From 0d420f361eabb4f11365d0a0580a803bff885309 Mon Sep 17 00:00:00 2001 From: Raphael Titsworth-Morin Date: Thu, 5 Sep 2024 14:20:24 -0700 Subject: [PATCH 1/2] initial postgres docs --- docs/concepts/configuration.md | 13 ++++++++++- docs/concepts/managed-storage.md | 39 ++++++++++++++++++++++++++++---- 2 files changed, 46 insertions(+), 6 deletions(-) diff --git a/docs/concepts/configuration.md b/docs/concepts/configuration.md index 0e890ea63..fa29b29e3 100644 --- a/docs/concepts/configuration.md +++ b/docs/concepts/configuration.md @@ -23,7 +23,7 @@ defang config set API_KEY You can use sensitive config by specifying them in the `environment` section of a service in a `compose.yaml` file without any value, or by specifying an environment key with a `null` value in your Pulumi code. -```ts +```yaml services: service1: image: image1:latest @@ -31,6 +31,17 @@ services: - API_KEY ``` +You can also use this syntax: + + +```yaml +services: + service1: + image: image1:latest + environment: + API_KEY: +``` + Use the `defang config` command of the Defang CLI to manage the values. ## Connecting Services diff --git a/docs/concepts/managed-storage.md b/docs/concepts/managed-storage.md index b522a6855..de20cbc25 100644 --- a/docs/concepts/managed-storage.md +++ b/docs/concepts/managed-storage.md @@ -19,7 +19,7 @@ Redis is an in-memory data structure store widely used for caching, real-time an To use managed Redis, in your `compose.yaml` file, use the `x-defang-redis` extension to define your Redis service. Adding the annotation will tell Defang to provision a managed instance, rather than running Redis as a service. Here's an example: ```yaml - redisx: + cache: image: redis:6.2 x-defang-redis: true restart: unless-stopped @@ -30,12 +30,41 @@ To use managed Redis, in your `compose.yaml` file, use the `x-defang-redis` exte ## Managed Postgres -:::info -As of July 22, 2024, managed Postgres is in development. -::: - Postgres, or PostgreSQL, is an advanced open-source relational database system known for its robustness, extensibility, and compliance with SQL standards, making it a popular choice for complex applications requiring reliable data integrity and sophisticated querying capabilities. +### How to use Managed Postgres + +To use managed Postgres, in your `compose.yaml` file, use the `x-defang-postgres` extension to define your Postgres service. Adding the annotation will tell Defang to provision a managed instance, rather than running Postgres as a service. + +#### Configuration + +When using managed Postgres, you must set a password for the database using `defang config set POSTGRES_PASSWORD`. If you do not provide the password, the deployment will fail. You can can assign the password in the service's environment variables as described in the [config section](./configuration.md). + +You can also optionally set the following, using `defang config set`: + +* `POSTGRES_USER`: The user for the managed Postgres instance. The default is `postgres`. +* `POSTGRES_DB`: The database name for the managed Postgres instance. The default is `postgres`. + +#### Connecting to Managed Postgres + +You can connect to the managed Postgres instance using the `POSTGRES_HOST`, `POSTGRES_USER`, and `POSTGRES_PASSWORD` environment variables. + +#### Example + +```yaml + database: + image: postgres:13 + x-defang-postgres: true + restart: unless-stopped + ports: + - mode: host + target: 5432 + environment: + // highlight-start + POSTGRES_PASSWORD: + // highlight-end +``` + ## Managed Object Storage :::info From 9fb1a7c1cffe555824d1ba686b963863b7decd5a Mon Sep 17 00:00:00 2001 From: Raphael Titsworth-Morin Date: Fri, 20 Sep 2024 15:44:24 -0700 Subject: [PATCH 2/2] update managed storage docs --- docs/concepts/configuration.md | 2 +- docs/concepts/defang-byoc.md | 2 +- docs/concepts/managed-storage.md | 72 ------------------- docs/concepts/managed-storage/_category_.json | 5 ++ .../managed-storage/managed-object-storage.md | 21 ++++++ .../managed-storage/managed-postgres.md | 65 +++++++++++++++++ .../concepts/managed-storage/managed-redis.md | 30 ++++++++ .../managed-storage/managed-storage.md | 10 +++ docs/providers/aws/_category_.json | 5 ++ docs/providers/{ => aws}/aws.md | 14 +++- docs/providers/azure.md | 2 +- docs/providers/digital-ocean/_category_.json | 5 ++ docs/providers/digital-ocean/digital-ocean.md | 11 +++ docs/providers/gcp.md | 2 +- docs/providers/playground.md | 9 +-- 15 files changed, 169 insertions(+), 86 deletions(-) delete mode 100644 docs/concepts/managed-storage.md create mode 100644 docs/concepts/managed-storage/_category_.json create mode 100644 docs/concepts/managed-storage/managed-object-storage.md create mode 100644 docs/concepts/managed-storage/managed-postgres.md create mode 100644 docs/concepts/managed-storage/managed-redis.md create mode 100644 docs/concepts/managed-storage/managed-storage.md create mode 100644 docs/providers/aws/_category_.json rename docs/providers/{ => aws}/aws.md (69%) create mode 100644 docs/providers/digital-ocean/_category_.json create mode 100644 docs/providers/digital-ocean/digital-ocean.md diff --git a/docs/concepts/configuration.md b/docs/concepts/configuration.md index fa29b29e3..5983eb03d 100644 --- a/docs/concepts/configuration.md +++ b/docs/concepts/configuration.md @@ -56,4 +56,4 @@ You can find a sample of how to set sensitive config values [here](https://githu Here are the different ways sensitive config values are stored depending on the provider you are using: - * [AWS](../providers/aws.md#secrets) + * [AWS](../providers/aws/aws.md#secrets) diff --git a/docs/concepts/defang-byoc.md b/docs/concepts/defang-byoc.md index e666ccd44..cd92ebf24 100644 --- a/docs/concepts/defang-byoc.md +++ b/docs/concepts/defang-byoc.md @@ -21,7 +21,7 @@ Defang does run the Defang build service to build your container images, which t The Public Beta of the v1 Defang BYOC AWS Provider is released as of Feb 1 2024. ::: -Please read the [AWS Provider](../providers/aws.md) documentation for more details about how the AWS provider works and how to get started. +Please read the [AWS Provider](../providers/aws/aws.md) documentation for more details about how the AWS provider works and how to get started. ## DigitalOcean diff --git a/docs/concepts/managed-storage.md b/docs/concepts/managed-storage.md deleted file mode 100644 index de20cbc25..000000000 --- a/docs/concepts/managed-storage.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -title: Managed Storage -description: Defang can help you provision managed storage services. -sidebar_position: 450 ---- - -# Managed Storage - -Defang helps you provision the infrastructure you need to run your services. That infrastructure is designed to scale in and out without persistent storage, so you can build highly scalable services. But Defang can also help you provision managed services to store and persist your data, like [caches](#managed-redis), databases[\*](#managed-postgres) , and object storage[\*](#managed-object-storage). - -## Managed Redis - -Redis is an in-memory data structure store widely used for caching, real-time analytics, and session management due to its high performance, low latency, and support for various data types. Defang can help you provision a managed Redis instance. To read more about the specifics of how this works for each provider, see the following: - - [Playground](../providers/playground#managed-redis) - - [AWS](../providers/aws#managed-redis) - -### How to use Managed Redis - -To use managed Redis, in your `compose.yaml` file, use the `x-defang-redis` extension to define your Redis service. Adding the annotation will tell Defang to provision a managed instance, rather than running Redis as a service. Here's an example: - -```yaml - cache: - image: redis:6.2 - x-defang-redis: true - restart: unless-stopped - ports: - - mode: host - target: 6379 -``` - -## Managed Postgres - -Postgres, or PostgreSQL, is an advanced open-source relational database system known for its robustness, extensibility, and compliance with SQL standards, making it a popular choice for complex applications requiring reliable data integrity and sophisticated querying capabilities. - -### How to use Managed Postgres - -To use managed Postgres, in your `compose.yaml` file, use the `x-defang-postgres` extension to define your Postgres service. Adding the annotation will tell Defang to provision a managed instance, rather than running Postgres as a service. - -#### Configuration - -When using managed Postgres, you must set a password for the database using `defang config set POSTGRES_PASSWORD`. If you do not provide the password, the deployment will fail. You can can assign the password in the service's environment variables as described in the [config section](./configuration.md). - -You can also optionally set the following, using `defang config set`: - -* `POSTGRES_USER`: The user for the managed Postgres instance. The default is `postgres`. -* `POSTGRES_DB`: The database name for the managed Postgres instance. The default is `postgres`. - -#### Connecting to Managed Postgres - -You can connect to the managed Postgres instance using the `POSTGRES_HOST`, `POSTGRES_USER`, and `POSTGRES_PASSWORD` environment variables. - -#### Example - -```yaml - database: - image: postgres:13 - x-defang-postgres: true - restart: unless-stopped - ports: - - mode: host - target: 5432 - environment: - // highlight-start - POSTGRES_PASSWORD: - // highlight-end -``` - -## Managed Object Storage - -:::info -As of July 22, 2024, managed object storage is planned for future development. -::: diff --git a/docs/concepts/managed-storage/_category_.json b/docs/concepts/managed-storage/_category_.json new file mode 100644 index 000000000..3a6426ea8 --- /dev/null +++ b/docs/concepts/managed-storage/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "Managed Storage", + "position": 450, + "collapsible": true +} \ No newline at end of file diff --git a/docs/concepts/managed-storage/managed-object-storage.md b/docs/concepts/managed-storage/managed-object-storage.md new file mode 100644 index 000000000..3c48f0931 --- /dev/null +++ b/docs/concepts/managed-storage/managed-object-storage.md @@ -0,0 +1,21 @@ +--- +title: Managed Object Storage +description: Defang can help you provision managed Redis instances. +sidebar_position: 3000 +--- + +# Managed Object Storage + +:::info Not Yet Supported +As of September 2024, Defang does not yet support managed Object Storage, but it is on our roadmap. If you are interested in Object Storage support, please vote on [this issue](https://github.com/DefangLabs/defang/issues/688). +::: + +Managed Object Storage, like AWS S3, is a service that allows you to store and retrieve large amounts of data. Object storage is ideal for storing unstructured data like images, videos, and backups. + +## Current Support + +| Provider | Managed Object Storage | +| --- | --- | +| [Playground](/docs/providers/playground.md) | ❌ | +| [AWS](/docs/providers/aws/aws.md) | ❌ | + diff --git a/docs/concepts/managed-storage/managed-postgres.md b/docs/concepts/managed-storage/managed-postgres.md new file mode 100644 index 000000000..262ed272b --- /dev/null +++ b/docs/concepts/managed-storage/managed-postgres.md @@ -0,0 +1,65 @@ +--- +title: Managed Postgres +description: Defang can help you provision managed Postgres. +sidebar_position: 1000 +--- + +# Managed Postgres + +Postgres, or PostgreSQL, is an advanced open-source relational database system known for its robustness, extensibility, and compliance with SQL standards, making it a popular choice for complex applications requiring reliable data integrity and sophisticated querying capabilities. + +## Current Support + +| Provider | Managed Postgres | +| --- | --- | +| [Playground](/docs/providers/playground) | ❌ | +| [AWS](/docs/providers/aws/aws.md#managed-postgres) | ✅ | + +## How to use Managed Postgres + +To use managed Postgres, in your `compose.yaml` file, use the `x-defang-postgres` extension to define your Postgres service. Adding the extension will tell Defang to provision a managed instance, rather than running Postgres as a service. + +### Required Configuration + +When using managed Postgres, you **must** set a password for the database using `defang config set POSTGRES_PASSWORD`. If you do not provide the password, the deployment will fail. You can can assign the password in the service's environment variables (to learn more about how this works, read about [configuration](../configuration.md)). + +### Optional Configuration + +You can also set the following environment variables to configure the managed Postgres instance: + +- `POSTGRES_USER`: The user for the managed Postgres instance. The default is `postgres`. +- `POSTGRES_DB`: The database name for the managed Postgres instance. The default is `postgres`. + +### Connecting to Managed Postgres + +You can connect to the managed Postgres instance using the name of your service as the hostname, `POSTGRES_USER`, `POSTGRES_DB`, and `POSTGRES_PASSWORD` environment variables. + +### Example + +```yaml + app: + # [...] + environment: + POSTGRES_HOST: database + POSTGRES_USER: postgres + POSTGRES_DB: postgres + // highlight-start + # Note: by leaving the value empty, Defang will use the + # value set using `defang config set POSTGRES_PASSWORD` + POSTGRES_PASSWORD: + // highlight-end + database: + image: postgres:15 + x-defang-postgres: true + restart: unless-stopped + ports: + - mode: host + target: 5432 + environment: + // highlight-start + # Note: by leaving the value empty, Defang will use the + # value set using `defang config set POSTGRES_PASSWORD` + POSTGRES_PASSWORD: + // highlight-end + +``` diff --git a/docs/concepts/managed-storage/managed-redis.md b/docs/concepts/managed-storage/managed-redis.md new file mode 100644 index 000000000..67d1c3276 --- /dev/null +++ b/docs/concepts/managed-storage/managed-redis.md @@ -0,0 +1,30 @@ +--- +title: Managed Redis +description: Defang can help you provision managed Redis instances. +sidebar_position: 2000 +--- + +# Managed Redis + +Redis is an in-memory data structure store widely used for caching, real-time analytics, and session management due to its high performance, low latency, and support for various data types. Defang can help you provision a managed Redis instance. + +## Current Support + +| Provider | Managed Redis | +| --- | --- | +| [Playground](/docs/providers/playground.md#managed-redis) | ❌ | +| [AWS](/docs/providers/aws/aws.md#managed-redis) | ✅ | + +## How to use Managed Redis + +To use managed Redis, in your `compose.yaml` file, use the `x-defang-redis` extension to define your Redis service. Adding the extension will tell Defang to provision a managed instance, rather than running Redis as a service. Defang will use the image tag to determine the version to provision from your cloud provider. Here's an example: + +```yaml +cache: + image: redis:6.2 + x-defang-redis: true + restart: unless-stopped + ports: + - mode: host + target: 6379 +``` diff --git a/docs/concepts/managed-storage/managed-storage.md b/docs/concepts/managed-storage/managed-storage.md new file mode 100644 index 000000000..1f3d6703d --- /dev/null +++ b/docs/concepts/managed-storage/managed-storage.md @@ -0,0 +1,10 @@ +--- +title: Managed Storage +description: Defang can help you provision managed storage services. +sidebar_position: 000 +--- + +# Managed Storage + +Defang helps you provision the infrastructure you need to run your services. That infrastructure is designed to scale in and out without persistent storage, so you can build highly scalable services. But Defang can also help you provision managed services to store and persist your data, like [caches](./managed-redis.md), [databases](./managed-postgres.md), and [object storage](./managed-object-storage.md). + diff --git a/docs/providers/aws/_category_.json b/docs/providers/aws/_category_.json new file mode 100644 index 000000000..1df8847e1 --- /dev/null +++ b/docs/providers/aws/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "AWS", + "position": 1000, + "collapsible": true +} \ No newline at end of file diff --git a/docs/providers/aws.md b/docs/providers/aws/aws.md similarity index 69% rename from docs/providers/aws.md rename to docs/providers/aws/aws.md index c661084e4..5d830dc47 100644 --- a/docs/providers/aws.md +++ b/docs/providers/aws/aws.md @@ -1,7 +1,7 @@ --- title: AWS description: Defang allows you to easily create and manage full, scalable applications with AWS. -sidebar_position: 010 +sidebar_position: 000 --- # AWS @@ -14,7 +14,7 @@ Why should you use Defang with AWS? Defang allows you to easily create and manag ## Getting Started -Getting started with the Defang BYOC AWS Provider is easy. First, make sure you [install the latest version of the Defang CLI](../getting-started#authenticate-with-defang). Then, make sure you have properly [authenticated your AWS account](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html). The Defang CLI will automatically check for environment variables like `AWS_PROFILE`, `AWS_REGION`, `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`. If they are set, the CLI will automatically use the Defang BYOC AWS Provider using the appropriate credentials. If you have credentials in the `~/.aws/credentials` file, but nothing configured in environment variables, you can use the `--provider=aws` flag to tell the Defang CLI to use the AWS Provider. +Getting started with the Defang BYOC AWS Provider is easy. First, make sure you [install the latest version of the Defang CLI](../../getting-started#authenticate-with-defang). Then, make sure you have properly [authenticated your AWS account](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html). The Defang CLI will automatically check for environment variables like `AWS_PROFILE`, `AWS_REGION`, `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`. If they are set, the CLI will automatically use the Defang BYOC AWS Provider using the appropriate credentials. If you have credentials in the `~/.aws/credentials` file, but nothing configured in environment variables, you can use the `--provider=aws` flag to tell the Defang CLI to use the AWS Provider. :::tip If you have the aws CLI installed, you should be able to successfully run `aws sts get-caller-identity` and see your account ID. @@ -44,6 +44,14 @@ To deploy your services, the Defang CLI packages your code and uploads it to an The provider runs your workloads using ECS using Fargate. It provisions a VPC with public and private subnets, and deploys your services to the private subnets. It then provisions an Application Load Balancer (ALB) and routes traffic to your services. +## Managed Storage + +Defang can help you provision [managed storage](/docs/concepts/managed-storage/managed-storage.md) services. The following managed storage services are supported on AWS: + +### Managed Postgres + +When using [Managed Postgres](/docs/concepts/managed-storage/managed-postgres.md), the Defang CLI provisions an RDS Postgres instance in your account. + ### Managed Redis -When using [Managed Redis](../concepts/managed-storage.md#managed-redis), the Defang CLI provisions an ElastiCache Redis cluster in your account. \ No newline at end of file +When using [Managed Redis](/docs/concepts/managed-storage/managed-redis.md), the Defang CLI provisions an ElastiCache Redis cluster in your account. \ No newline at end of file diff --git a/docs/providers/azure.md b/docs/providers/azure.md index 32e346ad1..f1142cc4f 100644 --- a/docs/providers/azure.md +++ b/docs/providers/azure.md @@ -1,7 +1,7 @@ --- title: Azure description: Defang will allow you to easily create and manage full, scalable applications with Azure. -sidebar_position: 100 +sidebar_position: 4000 --- # Azure diff --git a/docs/providers/digital-ocean/_category_.json b/docs/providers/digital-ocean/_category_.json new file mode 100644 index 000000000..eea74efc0 --- /dev/null +++ b/docs/providers/digital-ocean/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "Digital Ocean", + "position": 2000, + "collapsible": true +} \ No newline at end of file diff --git a/docs/providers/digital-ocean/digital-ocean.md b/docs/providers/digital-ocean/digital-ocean.md new file mode 100644 index 000000000..69d47e744 --- /dev/null +++ b/docs/providers/digital-ocean/digital-ocean.md @@ -0,0 +1,11 @@ +--- +title: Digital Ocean +description: Defang allows you to easily create and manage full, scalable applications with Digital Ocean. +sidebar_position: 000 +--- + +# Digital Ocean + +:::info +V1 Defang BYOC Digital Ocean Provider is in private beta as of September 2024. +::: diff --git a/docs/providers/gcp.md b/docs/providers/gcp.md index 816bf7a76..aeb072f88 100644 --- a/docs/providers/gcp.md +++ b/docs/providers/gcp.md @@ -1,7 +1,7 @@ --- title: GCP description: Defang will allow you to easily create and manage full, scalable applications with GCP. -sidebar_position: 200 +sidebar_position: 3000 --- # GCP diff --git a/docs/providers/playground.md b/docs/providers/playground.md index 1a666813b..38dd1496e 100644 --- a/docs/providers/playground.md +++ b/docs/providers/playground.md @@ -1,7 +1,7 @@ --- title: Playground description: The Defang Playground is a free tier that allows you to experiment with Defang. -sidebar_position: 000 +sidebar_position: 0000 --- The Defang Playground is a free tier that allows you to experiment with Defang. The Playground is a shared that cannot be used to run production workloads. The Playground is a great way to get started with Defang and to experiment with the Defang CLI. @@ -10,9 +10,4 @@ This page highlights architectural considerations when deploying to the playgrou ## Overview -Overall, the Defang Playground is very similar to deploying to your own cloud account. The Playground runs on a Defang-managed AWS account, so you can expect it to work similarly to deploying to [AWS](./aws.md). - -## Managed Redis - -The playground supports [Managed Redis](../concepts/managed-storage.md#managed-redis). Fundamentally, this works the same as it does in [AWS](./aws.md#managed-redis). - +Overall, the Defang Playground is very similar to deploying to your own cloud account. The Playground runs on a Defang-managed AWS account, so you can expect it to work similarly to deploying to [AWS](./aws/aws.md).