data.all
data-dot-all.github.io
- Free plan
- Web
- API
- Self-hosted
What data.all is
data.all is an open source platform deployed in AWS accounts to manage and discover data assets. Its centralized catalog inventories datasets, tables, folders, and dashboards, with filters for asset type, tags, region, and glossary terms. Datasets can contain structured tables or unstructured folders and use S3, KMS, IAM, and Glue resources. Teams can share datasets across environments by granting selected roles read access through Lake Formation without moving data. The catalog imports tables from Redshift Serverless and provisioned clusters. Analytics and machine-learning options include Athena worksheets, SageMaker Studio access, Jupyter notebooks, QuickSight dashboards, and HealthOmics Ready2Run workflows. Administrators manage teams and users across environments. Authentication can use Cognito federation with a corporate identity provider or an external OIDC provider, relying on identity-provider groups. Deployments may be internet-facing or VPC-facing. The AWS guide requires at least two accounts in matching regions and administrator credentials for tooling and deployment accounts. A Docker Compose local setup is documented, and modules can be enabled or disabled through configuration.
Who it suits
AWS teams that need a shared catalog, controlled data access, analytics tools, and machine-learning workspaces. It suits organizations able to operate multi-account AWS deployments.
Good to know, and what to watch
- Open source and free
- Catalogs datasets, tables, folders, and dashboards
- Lake Formation sharing avoids moving data
- Supports internet-facing and VPC-facing deployments
- Includes Athena, SageMaker, and QuickSight options
- Requires at least two AWS accounts
- Administrator credentials required during setup
- Selected AWS regions must match
- Some features depend on configuration
The verdict
data.all brings cataloging, sharing, analytics, and machine-learning access into an AWS-centered platform. Deployment prerequisites and administrator requirements make its infrastructure setup important to plan first.
What the maker publishes
- Administration
- Administrators can organize and manage teams and users across environments. source
- Analytics and ML
- The platform provides Athena worksheets, Jupyter notebooks through SageMaker, SageMaker Studio access, and QuickSight dashboards. source
- Audit
- For structured data, SQL queries from EMR, Redshift, Glue Jobs, and Athena are captured through Lake Formation. source
- AWS integrations
- The documented architecture uses AWS services including Cognito, CloudFront or an internal Application Load Balancer, API Gateway, Lambda, Aurora PostgreSQL, ECS Fargate, and OpenSearch. source
- Catalog
- Its centralized catalog inventories datasets, tables, folders, and dashboards, with search filters for asset type, tags, region, and glossary terms. source
- Configuration
- Modules and some features can be enabled or disabled through configuration, and the guide notes that file uploads are disabled in its example configuration. source
- Data catalog
- Its centralized catalog inventories datasets, tables, folders, and dashboards, with search filters for asset type, tags, region, and glossary terms. source
- Datasets
- Datasets can hold structured tables or unstructured folders and use AWS S3, KMS, IAM, and Glue resources. source
- Data sharing
- Dataset sharing across environments can grant selected team roles read access through Lake Formation without moving the data. source
- Data sharing
- Users can request access to another team’s data assets, and asset owners can accept or reject those requests. source
- Deployment
- The AWS deployment guide requires a tooling account and at least one deployment account, and their selected regions must match. source
- Deployment options
- The deployment guide describes internet-facing and VPC-facing architectures; the VPC-facing option makes the UI and APIs reachable only within the VPC. source
- Identity
- Authentication can use Amazon Cognito federated with a corporate identity provider, or an external OIDC provider; data.all relies on identity-provider group information. source
- Integrations
- Documented modules include AWS Glue, Lake Formation, Amazon S3, Athena, SageMaker, QuickSight, and HealthOmics Ready2Run workflows. source
- Local use
- The project documents a Docker Compose local setup with frontend, database, GraphQL, CDK proxy, and OpenSearch containers; AWS credentials are needed to create environments and datasets. source
- Network options
- Deployments can be internet facing, with public APIs protected by AWS WAF, or VPC facing, with private sites and APIs accessible through VPC endpoints. source
- Notebooks
- The platform provides notebooks based on Amazon SageMaker Studio for data scientists to build machine learning models. source
- Operational limit
- The AWS deployment instructions require administrator credentials for the tooling and deployment accounts during setup. source
- Purpose
- data.all is an open source platform that can be deployed in AWS accounts to help users manage and discover data assets. source
- Redshift integration
- It supports importing tables from Redshift Serverless and provisioned clusters into the catalog; connections can use Secrets Manager secrets or Redshift users. source
- Requirements
- AWS deployment requires at least two AWS accounts and administrator credentials for the tooling and deployment accounts. source
- Security
- Dataset S3 buckets are encrypted with AWS managed KMS keys, and data access is logged through CloudTrail. source
Checked against the maker's pages on 8 Oct 2026.