Execution of arbitrary code can be achieved via the safe_eval and safe_exec functions of the llama-index-experimental/llama_index/experimental/exec_utils.py Python file. The functions allow the user to run untrusted code via an eval or exec function while only permitting whitelisted functions. However, an attacker can leverage the whitelisted pandas.read_pickle function or other 3rd party library functions to achieve arbitrary code execution. This can be exploited in the Pandas Query Engine.
This potential attack vector is present in LlamaIndex v0.10.29 and newer in the llama-index-experimental package.
AV:L/AC:L/PR:N/UI:R/S:U/C:H/I:H/A:H
CWE-95: Improper Neutralization of Directives in Dynamically Evaluated Code (‘Eval Injection’)
The safe_exec and safe_eval functions both use the _verify_source_safety function to define whitelisted functions (see ALLOWED_IMPORTS constant variable below), and they also call a _get_restricted_globals function to disallow any imports that aren’t on the safe list.
This list introduces a substantial security risk, as outside of the first three, which are core Python libraries, the rest are 3rd party libraries containing functions that could be used maliciously. For example, pandas.read_pickle(malicious_url) can be used to load a remote malicious pickle file, which could lead to arbitrary code execution. Other functions such as pandas.read_csv() or pandas.Dataframe.to_csv() can read and write arbitrary files on the victim’s filesystem.
An example of where safe_eval and safe_exec are used is in the PandasQueryEngine. Its intended use is to allow a user to ask an LLM natural language questions about a pandas dataframe, and the LLM writes Python code based on the dataframe to answer the question and executes it using safe_eval and safe_exec. A malicious actor could abuse this by hosting a specially crafted dataframe on a website containing a hidden prompt that instructs the LLM to ignore its previous instructions and return a malicious statement to be evaluated (see below example). The malicious statement runs the pandas.read_pickle function on a remote URL (note that pandas must be represented by pd, as is common practice, because that’s how the library is passed to safe_eval). The URL can point to a malicious pickle file which is capable of performing arbitrary code execution when loaded.
When the query function is called on the malicious dataframe, an LLM is prompted to write a statement to be evaluated using numpy and pandas. The LLM is given the results of the DataFrame.head() function as context. The prompt instructs the LLM to output a line of code which loads a pickle from a URL. This output is then passed to the parse function of the PandasInstructionParser object. This function calls the default_output_processor function, which executes the safe_eval function on the given string, with the pandas and numpy packages allowed to be used.
def default_output_processor(
output: str, df: pd.DataFrame, **output_kwargs: Any
) -> str:
"""Process outputs in a default manner."""
import ast
import sys
import traceback
if sys.version_info < (3, 9):
logger.warning(
"Python version must be >= 3.9 in order to use "
"the default output processor, which executes "
"the Python query. Instead, we will return the "
"raw Python instructions as a string."
)
return output
local_vars = {"df": df}
global_vars = {"np": np, "pd": pd}
output = parse_code_markdown(output, only_last=True)[0]
# NOTE: inspired from langchain's tool
# see langchain.tools.python.tool (PythonAstREPLTool)
try:
tree = ast.parse(output)
module = ast.Module(tree.body[:-1], type_ignores=[])
safe_exec(ast.unparse(module), {}, local_vars) # type: ignore
module_end = ast.Module(tree.body[-1:], type_ignores=[])
module_end_str = ast.unparse(module_end) # type: ignore
if module_end_str.strip("'\"") != module_end_str:
# if there's leading/trailing quotes, then we need to eval
# string to get the actual expression
module_end_str = safe_eval(module_end_str, {"np": np}, local_vars)
try:
# str(pd.dataframe) will truncate output by display.max_colwidth
# set width temporarily to extract more text
if "max_colwidth" in output_kwargs:
pd.set_option("display.max_colwidth", output_kwargs["max_colwidth"])
output_str = str(safe_eval(module_end_str, global_vars, local_vars))
pd.reset_option("display.max_colwidth")
return output_str
except Exception:
raise
except Exception as e:
err_string = (
"There was an error running the output as Python code. "
f"Error message: {e}"
)
traceback.print_exc()
return err_string
The safe_eval function checks the string for disallowed actions, and if it’s clean, runs eval on the statement. Since pandas.read_pickle is an allowed action, the arbitrary code in the pickle file is executed.
