End Users
These are your customers and or team members who use your services available in your AI software application.
AI-as-a-Service
This layer contains tools that help integrate AI into your business. The primary users of these services are team members from your company, usually Data Scientists and Data Engineers.
AI Environment
The AI Environment layer includes all the tooling required for developing, hosting, and managing machine learning and AI models.
User Interface
Model inference is the stage where trained machine learning models are deployed and make guesses or decisions about new things it sees.
Docker is a platform that allows you to package your ML model, along with all its dependencies (libraries, code, settings), into a lightweight "container" that can run the same way on any machine. This makes it super useful for deploying models consistently — whether on your laptop, in the cloud, or in production. It helps ensure the model behaves the same everywhere and simplifies scaling, testing, and version control.
Ray.io is a framework that makes it easy to run ML models (and other compute-heavy tasks) across many machines at once. It helps you scale model inference, distribute workloads, and manage computing resources efficiently. For example, if you're handling lots of predictions at the same time or processing huge datasets, Ray can split the work across CPUs or GPUs to do it faster.
ML Ops
ML Ops is the practice of managing the end-to-end machine learning lifecycle—including experimentation, deployment, monitoring, and governance—to ensure models are reliable and scalable in production.
MLflow is an open-source platform used to manage the entire machine learning lifecycle, from development to deployment and beyond. It facilitates experiment tracking, packaging models for reuse, and deploying them to different platforms, all while providing a centralized model registry for versioning and collaboration.
Model Repository
A model repository securely stores and organizes machine learning models to support reuse, governance, and compliance.
Harbor is an open source registry that secures artifacts with policies and role-based access control, ensures images are scanned and free from vulnerabilities, and signs images as trusted.
Model Development
Model development is the process of designing, building, training, and refining machine learning models to solve specific problems using data.
The TensorFlow platform helps you implement best practices for data automation, model tracking, performance monitoring, and model retraining.
PyTorch is an open source machine learning (ML) framework based on the Python programming language and the Torch library. Torch is an open source ML library used for creating deep neural networks and is written in the Lua scripting language. It's one of the preferred platforms for deep learning research.
Jupyter allows users to create and share documents that combine code, equations, visualizations, and narrative text in a single document. This makes it ideal for tasks like exploring data, building and testing models, and creating reproducible research reports.
Data Environment
The Data Environment layer includes all tooling required for processing and serving data to the AI environment or external BI tooling.
SQL Query Engine
Database tools for AI included PostgreSQL and Trino.
PostgreSQL is a powerful, open-source object-relational database management system (ORDBMS) that is known for its reliability, feature robustness, and performance.
Trino enables users to query data from various sources (like data lakes and relational databases) using standard SQL, without needing to move or copy the data.
CI/CD Management
A pipeline is a structure to automate deployment of updated versions of applications.
Dagster is a client layer tool for managing and automating the Continuous Integration and Continuous Delivery for an application.
Workload Management
Workload management includes tools like Apache Spark and Ray.io. Apache Spark is used for real-time and batch processing of analytics on large datasets. Ray.io is an open-source framework for scaling AI and Python applications, providing a distributed computing platform that simplifies the development and deployment of large-scale machine learning (ML) and AI workloads.
Data Stream Management
Data stream management tools allow clients to optimize data traffic on their containers.
Redpanda and Kafka are client level tools to better manage access for end users to large amounts of data.
Data Governance
Devops need a way to control and manage access to data.
Metadata and Unity Catalog are tools that provide a single interface and centralized repository for all metadata assets, including governance, access control, auditing, and data lineage. This allows devops to administer data access policies across workspaces, simplifying access management.
Storage Management
DevOps teams are responsible for managing data across the platform, including both containers and their associated object storage.
Minio allows devops to manage large amounts of data (object storage) for AI systems and works with cloud infrastructure providers like AWS. An example of functionality in Minio is the ability to provision client access to data storage.
Contianer Management
A user friendly interface is a key factor in helping devops with managing virtual machines and containers.
Rancher is an example of a user friendly interface used for managing virtual machines and containers.
Contianer Automation
Container automation allows the platform to automatically scale up whatever resources are needed to maintain the applications running on the network.
Kubernetes is a platform used by devops to automate the deployment, scaling, and management of clusters for virtual machines. It also schedules containers to run on the clusters, optimizing resource utilization and ensuring application availability.
Infrastructure
This layer is maintained by IT specialists who make sure the servers and hardware are functioning as expected.
Data Storage
Data Storage is a service that allows the storage of data. This can be done with on-premise servers or remote servers. Examples of data storage providers are AWS, Google Cloud, and Microsoft Azure. In addition to data storage, these providers typically offer a suite of cloud services that power modern development and operations.
HIPPA
HITRUST
SOC 2
ISO 27001
GDPR
PCI DSS
FedRAMP
NIST
FISMA
