LiteLLM Supply Chain Attack: The Full Story

LiteLLM Attack Highlights Pervasive Supply Chain Risks

On March 24, 2026, a sophisticated cyberattack targeted the popular Python library LiteLLM, a critical component in the AI development landscape. Malicious actors, identified as the hacker group TeamPCP, compromised the library’s official repository on the Python Package Index (PyPI), releasing poisoned versions (1.82.7 and 1.82.8) that put millions of development environments and enterprise systems at risk of data leakage.

Key Takeaways

• The LiteLLM library, with nearly 100 million monthly downloads, was compromised via a supply chain attack.
• The attack exploited a vulnerability in the Trivy security scanner, which was part of LiteLLM’s CI/CD pipeline.
• Malicious code was injected, stealing PyPI release tokens and allowing the direct upload of compromised versions.
• The attack aimed to exfiltrate sensitive data, including SSH keys, cloud credentials, and cryptocurrency wallet information.
• A self-inflicted flaw in the malicious code inadvertently revealed the attack earlier than intended.
• The attack demonstrated advanced techniques for persistence and lateral movement, including Kubernetes exploitation.
• Critical steps for mitigation include immediate uninstallation of affected versions and comprehensive credential rotation.

Attack Overview

The attack vector was not a direct vulnerability within LiteLLM itself, but rather a compromise of the tools used in its development process. Specifically, the open-source security scanner Trivy, integrated into LiteLLM’s Continuous Integration/Continuous Deployment (CI/CD) pipeline, was tampered with by TeamPCP. This initial compromise allowed the attackers to inject malicious code and steal the PyPI release token when the CI/CD pipeline executed the compromised Trivy scanner.

Attack Timeline:

• March 19: TeamPCP compromised Trivy GitHub Action tags, embedding malicious code.
• March 23: The attackers gained access to the Checkmarx KICS security scanning tool, facilitating the next phase.
• March 24: LiteLLM’s CI/CD pipeline ran the compromised Trivy, leading to the theft of the PyPI release token. This enabled the attackers to bypass standard procedures and push malicious versions of LiteLLM directly to PyPI.

The attack was brought to light prematurely due to an error in the malicious code. Version 1.82.8 contained a file, litellm_init.pth, that repeatedly executed and caused memory exhaustion and system crashes on testing machines. This unintended consequence alerted developers to the breach much sooner than the attackers likely planned, preventing potentially more extensive damage.

The malicious code deployed by TeamPCP was designed with advanced capabilities, including stealth, a broad impact scope, and mechanisms for persistence and lateral movement. It operated in distinct stages:

The first stage focused on comprehensive information gathering. The script systematically scanned infected systems for sensitive data, including SSH private keys, Git configurations, shell history, cloud provider credentials (AWS/GCP/Azure), Kubernetes configurations, database passwords, and cryptocurrency wallet files and mnemonic phrases. Given LiteLLM’s function as a unified API access point for various AI models, it often stores API keys, making its compromise a direct gateway into an organization’s AI infrastructure.

The second stage involved encrypted data exfiltration. Collected data was encrypted using AES-256-CBC, with session keys protected by a 4096-bit RSA public key. This encrypted data was then archived and transmitted to a deceptive domain, models.litellm.cloud, registered just one day before the attack and unrelated to LiteLLM’s official infrastructure. Reports indicate approximately 300GB of compressed credentials, involving around 500,000 sensitive entries, were exfiltrated.

The third stage, aimed at persistence and lateral movement, posed the most significant threat. On local machines, the malware established a backdoor script (sysmon.py) in user directories and created a systemd service for persistence, ensuring it could remain active even after LiteLLM was uninstalled. In Kubernetes environments, the attackers exploited service account tokens to deploy privileged Pods across all nodes, enabling network-wide propagation and escalating a single host compromise into a cluster-wide security incident.

To further obscure their activities, the attackers reportedly employed malicious bots to flood messages and hijacked maintainer accounts to close GitHub issues, attempting to suppress any early detection.

Potential Risks

While PyPI has removed the compromised versions and lifted the quarantine, the ramifications of this supply chain attack are extensive and may continue to unfold. The primary concerns include:

The difficulty in completely removing persistent backdoors. Due to their integration with systemd services and hidden directories, users might mistakenly believe the threat is gone after uninstalling LiteLLM, unaware that the backdoor could continue to operate undetected, facilitating ongoing data leakage and providing an entry point for further intrusions.

The cascading effect of credential compromise. The exfiltration of approximately 500,000 credentials across critical areas like cloud services, databases, and CI/CD pipelines presents a severe risk. Attackers can leverage these credentials to infiltrate additional systems, creating a widespread “domino effect” of security breaches.

The propagation risk through dependency chains. As a widely used dependency in the AI ecosystem, LiteLLM is integrated into over 2,000 packages, including notable ones like DSPy, MLflow, and Open Interpreter. Many developers may have unknowingly incorporated the malicious version through indirect dependencies. This widespread “unintentional infection” poses a long-term risk, particularly for outdated containers or unpatched CI/CD pipelines that might still harbor compromised components.

This incident echoes the Trust Wallet security breach, where a backdoor was implanted in a popular cryptocurrency wallet’s browser extension, leading to significant fund theft. Although the Trust Wallet attack involved direct code modification rather than third-party package tampering, the exfiltration of cryptocurrency wallet data in the LiteLLM attack serves as a stark warning. Developers and users must act swiftly to secure assets and replace compromised keys to prevent similar losses.

Furthermore, organizations that fail to promptly rotate compromised cloud credentials and database passwords risk exposing core business data and potentially suffering system-wide compromises, leading to incalculable economic and reputational damage.

The TeamPCP group had previously expressed intentions to steal commercial secrets over the long term, and the LiteLLM attack appears to be a calculated move within a broader strategy to infiltrate the open-source ecosystem. This serves as a critical reminder that supply chain security is paramount and that any oversight can have devastating consequences.

Security Tips

To address the immediate threats posed by this attack and to bolster defenses against future incidents, both individual developers and enterprises must take decisive action:

1. Verify Infection Status:
   • Execute the command pip show litellm to check the installed version. If it is 1.82.7 or 1.82.8, uninstall it immediately using pip uninstall litellm.
   • Clear your package manager cache to remove any lingering malicious components. Use rm -rf ~/.cache/uv or pip cache purge.
2. Comprehensive Credential Rotation:
   • Assume all credentials within affected environments have been compromised.
   • Immediately rotate all sensitive credentials, including SSH keys, cloud provider access tokens, database passwords, and API keys.
   • For cryptocurrency wallets, this means immediately transferring assets to new wallets and generating new private keys and mnemonic phrases.
3. Standardize Dependency Management:
   • Pin dependency versions to known secure versions. It is recommended to lock LiteLLM to version 1.82.6 or an earlier secure release.
   • Enhance the security of your CI/CD pipelines. Regularly audit and update security tools, and implement robust checks to prevent exploitation by attackers.
   • Implement dependency scanning and vulnerability management tools to continuously monitor for compromised libraries.

Conclusion

The LiteLLM supply chain attack underscores the inherent vulnerabilities within the open-source ecosystem and highlights the critical importance of security in the rapidly advancing AI landscape. The integrity of core dependency libraries directly influences the stability and security of the entire ecosystem. Prioritizing supply chain security, conducting thorough risk assessments, and continuously strengthening defense mechanisms are essential to prevent significant data breaches and protect valuable assets.

About SlowMist

SlowMist, established in January 2018, is a leading blockchain security firm dedicated to enhancing the security of the global blockchain ecosystem. With a team possessing over a decade of network security expertise, SlowMist provides comprehensive security solutions, including security audits, threat intelligence, defense deployment, and consulting services. The firm has partnered with numerous prominent entities in the blockchain space, including HashKey Exchange, OSL, MEEX, Binance, OKX, and Crypto.com. SlowMist also offers specialized products like MistEye (Security Monitoring) and FireWall.x (Smart contract firewall), and its extensive work in cryptocurrency crime investigations has been recognized by international organizations and government bodies.

According to the portal: slowmist.medium.com