Is datasets safe?
What datasets does
datasets is a Hugging Face Python library that loads public datasets from the Hub with one load_dataset call. It also processes local data in formats such as CSV, JSON, Parquet, images, audio and PDF. The PyPI author field lists HuggingFace Inc., and the code lives in huggingface/datasets under Apache 2.0. It downloads data over the network, caches processed results on disk, can stream datasets without downloading them, and can read and write Hugging Face Storage Buckets. Before installing, set up a virtual environment, as the README advises, and check a dataset's size first or load it with streaming=True.
- Python filesystem read
- Python filesystem write/delete
- Python network egress
What to do: Nothing here argues against installing it. Grant the capabilities it lists only if you expect the tool to need them.
datasets is a PyPI package analyzed by SkillTotal's deterministic static scanner. The scan found no malicious indicators. It can: filesystem read, filesystem write and network egress — capabilities are what the code can do, not a verdict on intent. Risk score 0/100 (low).
datasets 5.2.0
Get notified when this component changes
Change alerts aren't live yet. Leave your email and we'll let you know when they launch. We use it only for this and delete it on request.
Automated static-analysis result. It can contain false positives and false negatives, and is not a claim about the intent of datasets's authors. Report a false positive.
Behavioral traits
How this component maps to the CSA agentic threat model. Descriptive — it never affects the risk score.
Findings (3)
The component reads files from disk.
with open(dataset_state_json_path, encoding="utf-8") as state_file:
with open(dataset_info_path, encoding="utf-8") as dataset_info_file:
legacy_dataset_info: dict = json.loads(fs.read_text(config.DATASETDICT_INFOS_FILENAME, encoding="utf-8")).get(
dataset_card = DatasetCard(fs.read_text(config.REPOCARD_FILENAME, newline="", encoding="utf-8"))
legacy_dataset_infos: dict = json.loads(fs.read_text(config.DATASETDICT_INFOS_FILENAME, encoding="utf-8"))
with open(path, encoding="utf-8") as f:
with open(path, encoding="utf-8") as f:
with open(names_filepath, encoding="utf-8") as f:
with self._open_with_fsspec().open() as f:
return self._open_with_fsspec().open()
with open(os.path.join(dataset_infos_dir, config.DATASETDICT_INFOS_FILENAME), encoding="utf-8") as f:
with open(standalone_yaml_path, encoding="utf-8") as f:
with open(os.path.join(self.path, config.DATASETDICT_INFOS_FILENAME), encoding="utf-8") as f:
with open(standalone_yaml_path, encoding="utf-8") as f:
with open(dataset_infos_path, encoding="utf-8") as f:
dataset_card_data = DatasetCard(hffs.read_text(readme_path, newline="", encoding="utf-8")).data
standalone_yaml_data = yaml.safe_load(hffs.read_text(standalone_yaml_path, newline="", encoding="utf-8"))
with open(file, "rb") as f:
with open(file, "rb") as f:
or json.loads(Path(cached_directory_path, "dataset_info.json").read_text(encoding="utf-8"))["config_name"]
or json.loads(Path(_cached_directory_path, "dataset_info.json").read_text(encoding="utf-8"))["config_name"]
with open(file, "rb") as f:
with open(file, encoding=self.config.encoding, errors=self.config.encoding_errors) as f:
with open(sample_file) as f:
with open(file, "rb") as f, TextIOWrapper(f, encoding="utf-8", newline="") as fp:
Why it matters: Usually legitimate, but worth confirming it can't be steered into reading sensitive files.
Fix: Confirm which files are read and that paths cannot be influenced by untrusted input to reach sensitive locations.
The component writes or deletes files on disk.
os.remove(file_path)
os.remove(tmp_file.name)
shutil.move(tmp_file.name, cache_file_name)
os.remove(tmp_file.name)
shutil.move(tmp_file.name, indices_cache_file_name)
shutil.rmtree(dirname)
shutil.move(tmp_dir, dirname)
shutil.rmtree(tmp_dir)
os.remove(path)
shutil.move(fs._strip_protocol(src), fs._strip_protocol(dst))
shutil.rmtree(self.name)
with open(dataset_infos_path, "w", encoding="utf-8") as f:
shutil.copytree(self.cache_dir, output_dir)
path.write_bytes(data)
open(probe_file, "a")
shutil.move(file, dest)
with open(fname, "wb") as stream:
with open(output_path, "wb") as extracted_file:
shutil.copyfileobj(gzip_file, extracted_file)
with open(output_path, "wb") as extracted_file:
shutil.copyfileobj(compressed_file, extracted_file)
with open(input_path, "rb") as ifh, open(output_path, "wb") as ofh:
with open(output_path, "wb") as extracted_file:
shutil.copyfileobj(compressed_file, extracted_file)
with open(output_path, "wb") as extracted_file:
Why it matters: Usually legitimate, but worth confirming the paths can't be controlled by untrusted input.
Fix: Confirm which files are written/deleted and that paths cannot be influenced by untrusted input.
The component makes outbound network requests.
import urllib
from aiohttp import ClientError
Why it matters: Usually legitimate, but confirm the destinations are expected and no sensitive data leaves.
Fix: Confirm the destination hosts are expected and that no sensitive data is sent off-host.
Maintain datasets?
The free Claude Code plugin scans each npm, PyPI or MCP package with this report's open-source engine before the install command runs. It runs on your machine and needs no account.
Check your own component
Run the same evidence-backed scan on any MCP server, agent skill, or package.
Scan your own componentHow we determine this: deterministic static analysis (regex + AST), evidence-anchored, no code execution. Methodology →