Open-Source AI in the Enterprise: Insights from a Survey of IT Leaders

AI and machine learning have become the engines of enterprise innovation, and open-source tools are the fuel. Whether it’s speeding up development, cutting costs, or unlocking new capabilities, open source is powering a shift in how businesses approach AI. But as these tools become more integral, they present new challenges. Companies must work to keep systems secure and manage performance at scale while ensuring teams have the skills to make the most of them.

Our latest report, The State of Enterprise Open-Source AI, dives into how enterprises are balancing these forces. We’ve sifted through survey data from 100 IT decision-makers at large and midsize enterprises to shed light on the trends, challenges, and opportunities shaping the future of AI in the enterprise. 

“Open-source AI is reshaping how enterprises innovate, offering tools that drive smarter decision-making and better operational efficiency. But as adoption grows, so does the need for strategies that balance experimentation with long-term stability and security,” says Peter Wang, Chief AI & Innovation Officer at Anaconda. “The challenge lies in building systems that not only solve today’s problems but scale seamlessly for the future.”

Find highlights from the research below.

Open-Source AI Adoption: The Current Landscape

Open-source AI is becoming the backbone of enterprise innovation, with adoption rates reflecting its growing importance across industries. These findings spotlight sectors leading the charge and tools setting the standard.

• Reliance on Open Source: Over half (58%) of organizations use open-source components in at least half of their AI/ML projects, with a third (34%) using them in three-quarters or more.
• Sector-Specific Trends: Adoption is strongest in sectors like education, manufacturing, retail, and financial services, while healthcare and IT exhibit lower reliance.

Security: The Critical Challenge

While open-source tools unlock innovation, they also come with security risks that can threaten enterprise stability and reputation. The data reveals the vulnerabilities organizations face and the steps businesses are taking to safeguard their systems. Addressing these challenges is vital for building trust and ensuring the safe deployment of AI/ML models.

• Incident Frequency: One-third (32%) of respondents experienced security vulnerabilities from open-source AI, with half of these incidents rated as “very” or “extremely” significant.
• Severity of Risks: The accidental installation of malicious code, while rare (10%), was the most severe, with 60% of incidents rated as highly significant.
• Proactive Measures: 61% of organizations use third-party software to scan for vulnerabilities, while 53% implement thorough manual code reviews.

Scaling AI with Confidence

Scaling AI is a balancing act: enterprises must handle increasing complexity without sacrificing performance or stability. The State of Enterprise Open-Source AI report uncovers the most pressing challenges organizations face when scaling AI initiatives and highlights best practices for maintaining reliability as AI systems grow in scope and ambition.

• Top Challenges: Scaling AI poses issues like fine-tuning large language models (LLMs) (41%) and managing computational resources (36%).
• Best Practices: 64% of respondents ensure code reproducibility across environments, and 59% maintain system stability while scaling.
• Performance Ratings: More than half (54%) rate their AI/ML stack as effective in scaling without increasing security risks.

The Future of Enterprise AI

These highlights are just the beginning. Access the full State of Enterprise Open-Source AI report for a deeper look at the data, including breakdowns on RAG and LLM implementation and cross-team collaboration in open source. Plus, find practical strategies and detailed insights to help you realize the ROI of open-source AI at your enterprise.

Download the full report now to uncover the strategies driving AI success across industries.