-
Notifications
You must be signed in to change notification settings - Fork 10
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Jacob Knightley
authored and
Jacob Knightley
committed
Jan 7, 2025
1 parent
6a09db7
commit 9c394f4
Showing
22 changed files
with
284 additions
and
240 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
$ErrorActionPreference = "Stop" | ||
|
||
# Define the paths relative to the script location | ||
$rootDir = (Resolve-Path "$PSScriptRoot\..\..\").Path | ||
$buildDir = "$rootDir\.readthedocs\_build" | ||
$docsDir = "$rootDir\.readthedocs\docs" | ||
$confDir = "$rootDir\.readthedocs\config" | ||
$requirements = "$confDir\requirements.txt" | ||
$packageRequirements = "$rootDir\requirements.txt" | ||
|
||
# Remove the contents of the build directory if it exists | ||
if (Test-Path -Path $buildDir) { | ||
Remove-Item -Recurse -Force $buildDir/* | ||
} | ||
|
||
# Install the requirements | ||
pip install -r $requirements | ||
|
||
# Install the package requirements | ||
pip install -r $packageRequirements | ||
|
||
# Run the Sphinx build | ||
sphinx-build -b html $docsDir $buildDir -c $confDir | ||
|
||
# Serve the built documentation | ||
Start-Process -NoNewWindow -FilePath "python" -ArgumentList "-m http.server --directory `"$buildDir`"" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
import os | ||
import sys | ||
|
||
# Set location of importable modules | ||
root_path = str(os.path.abspath(os.path.join(os.path.dirname(__file__), "..", ".."))) | ||
sys.path.insert(0, root_path + "/src") | ||
sys.path.insert(0, root_path + "/sample") | ||
|
||
# Configuration file for the Sphinx documentation builder. | ||
|
||
project = "fabric-cicd" | ||
copyright = "Microsoft Corporation" | ||
author = "Microsoft Corporation" | ||
|
||
# Load the version | ||
with open(root_path + "/VERSION") as version_file: | ||
release = version_file.read().strip() | ||
|
||
extensions = [ | ||
"sphinx.ext.autodoc", | ||
"sphinx_autodoc_typehints", | ||
"myst_parser", | ||
] | ||
|
||
html_theme = "sphinx_rtd_theme" | ||
|
||
autodoc_default_options = { | ||
"members": True, | ||
"show-inheritance": True, | ||
"special-members": "__init__", | ||
} | ||
|
||
suppress_warnings = ["myst.header"] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
# Read the Docs configuration file for Sphinx projects | ||
|
||
version: 2 | ||
|
||
build: | ||
os: ubuntu-22.04 | ||
tools: | ||
python: "3.10" | ||
|
||
sphinx: | ||
configuration: conf.py | ||
|
||
python: | ||
install: | ||
- requirements: requirements.txt | ||
- requirements: ../../requirements.txt |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
sphinx | ||
sphinx_rtd_theme | ||
sphinx_autodoc_typehints | ||
myst_parser |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
fabric_cicd | ||
========================= | ||
|
||
.. toctree:: | ||
:maxdepth: 4 | ||
:caption: Contents: | ||
|
||
overview | ||
library_reference/index | ||
installation | ||
usage |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
Installation | ||
============ | ||
|
||
To install fabric-cicd, run: | ||
|
||
.. code-block:: bash | ||
pip install fabric-cicd |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
fabric_workspace Module | ||
========================= | ||
|
||
.. automodule:: fabric_cicd.fabric_workspace | ||
:members: | ||
:undoc-members: | ||
:show-inheritance: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
Library Reference | ||
================= | ||
|
||
.. toctree:: | ||
:maxdepth: 2 | ||
|
||
fabric_workspace | ||
publish |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
publish Module | ||
================= | ||
|
||
.. automodule:: fabric_cicd.publish | ||
:members: | ||
:undoc-members: | ||
:show-inheritance: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
Overview | ||
========================= | ||
|
||
.. include:: ../../README.md | ||
:parser: myst_parser.sphinx_ | ||
:start-line: 12 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
Usage | ||
===== | ||
|
||
Authenticate with Default Credentials | ||
-------------------------------------- | ||
|
||
.. literalinclude:: ../../sample/auth_default_credential.py | ||
:language: python | ||
|
||
Authenticate with SPN Credentials | ||
-------------------------------------- | ||
|
||
.. literalinclude:: ../../sample/auth_spn_secret.py | ||
:language: python | ||
|
||
Setting Variables within Azure DevOps Release | ||
---------------------------------------------- | ||
|
||
.. literalinclude:: ../../sample/set_vars_from_ado_build.py | ||
:language: python | ||
|
||
Setting Variables within Local GIT Branch | ||
---------------------------------------------- | ||
|
||
.. literalinclude:: ../../sample/set_vars_from_git_branch.py | ||
:language: python | ||
|
||
Using Optional Parameters | ||
---------------------------------------------- | ||
|
||
.. literalinclude:: ../../sample/optional_parameters.py | ||
:language: python |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,81 +1,131 @@ | ||
# Fabric CICD | ||
[![PyPI version](https://badge.fury.io/py/fabric-cicd.svg)](https://badge.fury.io/py/fabric-cicd) | ||
|
||
[![Language](https://img.shields.io/badge/language-Python-blue.svg)](https://www.python.org/) | ||
[![TestPyPI version](https://img.shields.io/badge/TestPyPI-fabric--cicd-blue)](https://test.pypi.org/project/fabric-cicd/) | ||
[![Read The Docs](https://readthedocs.org/projects/fabric-cicd/badge/?version=latest&style=flat)](https://readthedocs.org/projects/fabric-cicd/) | ||
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black) | ||
[![Downloads](https://static.pepy.tech/badge/fabric-cicd)](https://pepy.tech/project/fabric-cicd) | ||
[![Code style: ruff](https://img.shields.io/badge/code%20style-ruff-000000.svg)](https://github.com/charliermarsh/ruff) | ||
|
||
<!-- [![PyPI version](https://badge.fury.io/py/fabric-cicd.svg)](https://badge.fury.io/py/fabric-cicd) --> | ||
<!-- [![Downloads](https://static.pepy.tech/badge/fabric-cicd)](https://pepy.tech/project/fabric-cicd) --> | ||
|
||
[Read full documentation on ReadTheDocs](https://fabric-cicd.readthedocs.io/en/latest/) | ||
|
||
--- | ||
[Read the documentation on ReadTheDocs!](https://fabric-cicd.readthedocs.io/en/stable/) | ||
--- | ||
|
||
Fabric CICD is a Python library designed for use with [Microsoft Fabric](https://learn.microsoft.com/en-us/fabric/) workspaces. This library is intended to support code first CICD integrations to seamlessly integrate Source Controlled workspaces into a deployment framework. The goal of this library is to support the CICD developers that don't want to dig into the weeds of interacting directly with the Microsoft Fabric APIs. | ||
## Project Overview | ||
|
||
Fabric CICD is a Python library designed for use with [Microsoft Fabric](https://learn.microsoft.com/en-us/fabric/) workspaces. This library is intended to support code first CICD integrations to seamlessly integrate Source Controlled workspaces into a deployment framework. The goal of this library is to support the CICD developers that don't want to dig into the weeds of interacting directly with the Microsoft Fabric APIs. | ||
Fabric CICD is a Python library designed for use with [Microsoft Fabric](https://learn.microsoft.com/en-us/fabric/) workspaces. This library is intended to support code first CICD integrations to seamlessly integrate Source Controlled workspaces into a deployment framework. The goal of this library is to support the CICD developers that don't want to dig into the weeds of interacting directly with the Microsoft Fabric APIs. | ||
|
||
If you encounter any issues, please [raise a bug](https://github.com/microsoft/fabric-cicd/issues/new?assignees=&labels=&projects=&template=bug_report.md&title=). | ||
|
||
If you have ideas for new features/functions, please [request a feature](https://github.com/microsoft/fabric-cicd/issues/new?assignees=&labels=&projects=&template=feature_request.md&title=). | ||
|
||
## Featured Scenarios | ||
### Featured Scenarios | ||
|
||
- Deploying Items hosted in a repository | ||
- 100% deployment, does not consider diffs | ||
- Deploys into the tenant of the executing identity | ||
|
||
### In Scope Item Types | ||
- Notebooks | ||
- Data Pipelines | ||
- Environments | ||
|
||
|
||
- Notebooks | ||
- Data Pipelines | ||
- Environments | ||
|
||
|
||
- Notebooks | ||
- Data Pipelines | ||
- Environments | ||
|
||
### Limitations | ||
- Notebooks Limitations | ||
- Attached lakehouses are not changed during the deployment | ||
- Attached resources are not included in the deployment | ||
- Data Pipelines | ||
- Connections are not changed during the deployment | ||
- Environments | ||
- Custom and Public Libraries are not included in the deployment | ||
- Folders | ||
- Sub folders are not included in the deployment | ||
|
||
## How to Use | ||
```shell | ||
pip install fabric-cicd | ||
``` | ||
|
||
```python | ||
from fabric_cicd import FabricWorkspace, publish_all_items, unpublish_all_orphan_items | ||
|
||
# Initialize the FabricWorkspace object with the required parameters | ||
target_workspace = FabricWorkspace( | ||
workspace_id='', | ||
environment='', | ||
repository_directory=r'', | ||
item_type_in_scope=[] | ||
) | ||
|
||
# Publish all items defined in item_type_in_scope | ||
publish_all_items(target_workspace) | ||
|
||
# Unpublish all items defined in item_type_in_scope not found in repository | ||
unpublish_all_orphan_items(target_workspace) | ||
``` | ||
|
||
## Contributing | ||
|
||
This project welcomes contributions and suggestions. Most contributions require you to agree to a | ||
Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us | ||
the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com. | ||
|
||
When you submit a pull request, a CLA bot will automatically determine whether you need to provide | ||
a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions | ||
provided by the bot. You will only need to do this once across all repos using our CLA. | ||
|
||
This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). | ||
For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or | ||
contact [[email protected]](mailto:[email protected]) with any additional questions or comments. | ||
|
||
- **Notebooks**: | ||
- Attached lakehouses are not changed during deployment | ||
- Attached resources are not included in deployment | ||
- **Data Pipelines**: | ||
- Connections are not changed during deployment | ||
- **Environments**: | ||
- Custom and public libraries are not included in deployment | ||
- **Folders**: | ||
- Sub folders are not included in deployment | ||
|
||
### Authentication | ||
|
||
- Optionally provide your own credential object aligned with the [TokenCredential class](https://learn.microsoft.com/en-us/dotnet/api/azure.core.tokencredential) in the `token_credential` parameter. | ||
- If the `token_credential` parameter is omitted, the library uses the Azure SDK `DefaultAzureCredential` for authentication. | ||
- Refer to the [Azure SDK documentation](https://learn.microsoft.com/en-us/azure/developer/python/sdk/authentication/credential-chains?tabs=dac#defaultazurecredential-overview) for the order in which credential types are attempted. | ||
- For local development with a UPN, install either the [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli-windows) or [Az.Accounts](https://www.powershellgallery.com/packages/Az.Accounts/2.2.3). | ||
- Note: When no credential is provided, the `DefaultAzureCredential` may choose an unexpected identity. For instance, if you log in to the Azure CLI with an SPN but log in to Az.Account with a UPN, the `DefaultAzureCredential` will choose the CLI authentication first. | ||
|
||
## Contribute | ||
|
||
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit [https://cla.opensource.microsoft.com](https://cla.opensource.microsoft.com). | ||
|
||
When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA. | ||
|
||
This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [[email protected]](mailto:[email protected]) with any additional questions or comments. | ||
|
||
### Requirements | ||
|
||
Before you begin, ensure you have the following installed: | ||
|
||
- [Python](https://www.python.org/downloads/) (version 3.10 or higher) | ||
- [PowerShell](https://docs.microsoft.com/en-us/powershell/scripting/install/installing-powershell) | ||
- [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli-windows) or [Az.Accounts PowerShell module](https://www.powershellgallery.com/packages/Az.Accounts/2.2.3) | ||
- [Visual Studio Code (VS Code)](https://code.visualstudio.com/) | ||
|
||
### Initial Configuration | ||
|
||
1. Clone the repository: | ||
|
||
```sh | ||
git clone https://github.com/microsoft/fabric-cicd.git /your/target/directory | ||
cd /your/target/directory | ||
``` | ||
|
||
1. Create a virtual environment: | ||
|
||
```sh | ||
python -m venv venv | ||
``` | ||
|
||
1. Activate the virtual environment: | ||
|
||
- On Windows: | ||
|
||
```sh | ||
.\venv\Scripts\activate | ||
``` | ||
|
||
- On macOS and Linux: | ||
|
||
```sh | ||
source venv/bin/activate | ||
``` | ||
|
||
1. Install the dependencies: | ||
|
||
```sh | ||
pip install -r requirements.txt | ||
pip install -r requirements-dev.txt | ||
``` | ||
|
||
1. Open the project in VS Code and ensure the virtual environment is selected: | ||
|
||
- Open the Command Palette (Ctrl+Shift+P) and select `Python: Select Interpreter`. | ||
- Choose the interpreter from the venv directory. | ||
## Support | ||
### How to file issues and get help | ||
This project uses GitHub Issues to track bugs, feature requests, and questions. Please search the existing issues before filing new issues to avoid duplicates. For new issues, file your bug, feature request, or question as a new Issue. | ||
### Microsoft Support Policy | ||
Support for this **PROJECT or PRODUCT** is limited to the resources listed above. | ||
## Trademarks | ||
This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft | ||
trademarks or logos is subject to and must follow | ||
[Microsoft's Trademark & Brand Guidelines](https://www.microsoft.com/en-us/legal/intellectualproperty/trademarks/usage/general). | ||
Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. | ||
Any use of third-party trademarks or logos are subject to those third-party's policies. | ||
This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow [Microsoft's Trademark & Brand Guidelines](https://www.microsoft.com/en-us/legal/intellectualproperty/trademarks/usage/general). Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies. |
Oops, something went wrong.