Dagster project file reference
This reference contains details about default and configuration files in a Dagster project, including:
- Names of files and directories and their purposes
- Recommended locations of files in a project directory
- Scenario-specifc best practices for structuring your Dagster project
Default files
The following demonstrates a Dagster project using the default project skeleton, generated by the create-dagster project
command:
- uv
- pip
.
└── my-project
├── pyproject.toml
├── src
│ └── my_project
│ ├── __init__.py
│ ├── definitions.py
│ └── defs
│ └── __init__.py
├── tests
│ └── __init__.py
└── uv.lock
.
└── my-project
├── pyproject.toml
├── src
│ └── my_project
│ ├── __init__.py
│ ├── definitions.py
│ └── defs
│ └── __init__.py
└── tests
└── __init__.py
To use tree
, install it with brew install tree
(Mac), or follow the installation instructions.
- The Python package
my_project
lives insrc/my_project
and contains the deployable code that defines your Dagster pipelines. my_project/definitions.py
is the entry point that Dagster will load when deploying your code location.src/my_project/defs
will contain your Dagster definitions and component definitions.src/my_project/components
(not pictured) is an optional folder used to define custom components, and optionally other code you wish to share across Dagster definitions.tests
is a separate Python package defined at the top level (outsidesrc
). It should contain tests for themy_project
package.pyproject.toml
is a standard Python package configuration file. In addition to the regular Python package metadata, it contains atool.dg
section fordg
-specific settings.
Configuration files
Depending on your use case or if you're using Dagster+, you may also need to add additional configuration files to your project. To see how these files might fit into your project, see the Deployment-specific project structures section.
File/Directory | Description | OSS | Dagster+ |
---|---|---|---|
dagster.yaml | Configures your Dagster instance, including defining storage locations, run launchers, sensors, and schedules. For more information. including a list of use cases and available options, see the dagster.yaml reference.For Dagster+ Hybrid deployments, this file can be used to customize the Hybrid agent. | Optional | Optional |
dagster_cloud.yaml | Defines code locations for Dagster+. For more information, see the dagster_cloud.yaml reference. | Not applicable | Recommended |
deployment_settings.yaml | Configures settings for full deployments in Dagster+, including run queue priority and concurrency limits. Refer to the Deployment settings reference for more info. Note: This file can be named anything, but we recommend choosing an easily understandable name. | Not applicable | Optional |
workspace.yaml | Defines multiple code locations for local development or deploying to your infrastructure. For more information and available options, see the workspace.yaml file reference | Optional | Not applicable |
Deployment-specific project structures
Using the default project skeleton, let's take a look at how some example Dagster projects would be structured in the following deployment scenarios:
With the exception of dagster_cloud.yaml
, it's not necessary for configuration files to be located with your project files. These files typically need to be located in DAGSTER_HOME
. For example, in larger deployments, DAGSTER_HOME
and Dagster infrastructure configuration can be managed separately from the code locations they support.
Local development
- Single code location
- Multiple code locations
For local development, a project with a single code location might look like this:
.
└── my-project
├── dagster.yaml ## optional, used for instance settings
├── pyproject.toml
├── src
│ └── my_project
│ ├── __init__.py
│ ├── definitions.py
│ └── defs
│ └── __init__.py
├── tests
│ └── __init__.py
└── uv.lock
For local development, a workspace with multiple projects that each define a different code location might look like this:
.
├── dagster.yaml ## optional, used for instance settings
├── deployments
│ └── local
│ ├── pyproject.toml
│ └── uv.lock
├── dg.toml ## defines multiple projects
└── projects
├── project-1
│ ├── pyproject.toml
│ ├── src
│ │ └── project_1
│ │ ├── __init__.py
│ │ ├── definitions.py
│ │ └── defs
│ │ └── __init__.py
│ ├── tests
│ │ └── __init__.py
│ └── uv.lock
└── project-2
├── pyproject.toml
├── src
│ └── project_2
│ ├── __init__.py
│ ├── definitions.py
│ └── defs
│ └── __init__.py
├── tests
│ └── __init__.py
└── uv.lock
For more information on creating a Dagster workspace with multiple projects, see Managing multiple projects with workspaces.