diff --git a/README.md b/README.md index 1ca986f..303b8c6 100644 --- a/README.md +++ b/README.md @@ -13,3 +13,40 @@ which hosts binary wheels for a variety of Python modules. For more informatiion about RISE, visit the [project website](https://riseproject.dev/). + +## Motivation + +Python is commonly used in the fields of scientific computing, data analysis and +machine learning. However, the Python packages used in these disciplines aren’t +wholly written in Python - they also contain a lot of code written in C/C++ or +other languages which needs to be built as part of the module. Such projects +typically create binary wheels for each of their releases and upload these +wheels to pypi, the Python Package Index. This allows users to easily and +quickly install tested, prebuilt versions of their favourite projects using the +pip (or [uv](https://docs.astral.sh/uv/)) tool. + +Until recently, Python packaging infrastructure like +[auditwheel](https://github.com/pypa/auditwheel), +[cibuildwheel](https://github.com/pypa/cibuildwheel), and +[manylinux](https://github.com/pypa/manylinux) did not support riscv64, and no +native runners for GitHub Actions were available. This made supporting the +architecture difficult for open-source projects without complicated build +processes and emulated systems. However, the aforementioned infrastructure now +supports riscv64, and with the RISE [RISC-V +Runners](https://riscv-runners.riseproject.dev/) project, maintainers have the +option of building binary wheels on native riscv64 platforms. The RISE Python +Wheels project's goal is to accelerate this adoption and ensure that the riscv64 +architecture is fully-supported for data science and machine learning +applications. + +## Installing the Packages + +Under Construction + +## Contributing + +Under Construction + +## Infrastructure + +See [infrastructure.md](docs/infrastructure.md) diff --git a/docs/infrastructure.md b/docs/infrastructure.md new file mode 100644 index 0000000..04c4a0d --- /dev/null +++ b/docs/infrastructure.md @@ -0,0 +1,58 @@ +# Infrastructure + +The [python-wheels](https://github.com/riseproject-dev/python-wheels) project +makes use of some supplemental infrastructure available both on GitHub, and +externally. The following sections outline these services and how to +configure/maintain them. + +## RISE RISC-V Runners + +For RISE-hosted builds of Python wheels and upstream submissions, the +`python-wheels` repository uses the [RISC-V +Runners](https://riscv-runners.riseproject.dev/) project. It is already enabled +for the repository; to reconfigure, see the [installation +guide](https://riscv-runners.riseproject.dev/docs/getting-started/install). + +## GitLab + +The `python-wheels` project is a continuation of +[wheel_builder](https://riseproject.gitlab.io/python/wheel_builder/index.html), +which is hosted on GitLab. This project leverages the GitLab CI/CD framework to +create pipelines for building, testing, and deploying wheels, along with +automatic documentation updates and periodic checks for new package versions +upstream. + +While development and maintenance now primarily occur on GitHub, the existing +GitLab infrastructure (particularly the package registry) must be maintained. + +### GitLab for Open Source + +We apply for the [GitLab for Open +Source](https://about.gitlab.com/solutions/open-source/join/) program. This +provides us with the GitLab Ultimate Plan at no cost. Membership requires annual +renewal. + +### CI/CD Pipeline Tokens and Access + +The GitLab CI/CD pipelines make use of project tokens, generated at different +project scopes. Two tokens are defined: + +1. `PYTHON_CI_GROUP_TOKEN` - Defined at the "Group" + ([Python](https://gitlab.com/riseproject/python)) level. Used for triggering + pipelines for merge requests and most automatic updates and checks. The value + of this token is inherited by `wheel_builder` as `PYTHON_CI_GROUP_TOKEN` + under `Settings -> CI/CD Settings -> Variables` (see the [CI/CD variables + panel](https://gitlab.com/riseproject/python/wheel_builder/-/settings/ci_cd#js-cicd-variables-settings)). +2. `rise-github-package-registry` - Defined at the "Repository" + ([wheel_builder](https://gitlab.com/riseproject/python/wheel_builder/-/settings/repository#js-deploy-tokens)) + level. Used for pushing wheels to the GitLab package registry from GitHub. + +These tokens are generated with a 1-year expiry timeline, so they need to be +regenerated by a project administrator/owner. + +#### Token Scope Reference + +| Token | Level | Role | Scopes | Used For | Expiry | +|-------|-------|------|--------|----------|--------| +| `PYTHON_CI_GROUP_TOKEN` | Group ([Python](https://gitlab.com/riseproject/python)) | Maintainer | `api`, `read_api`, `read_repository`, `write_repository` | MR pipeline triggers, auto updates/checks | 1 year (Unknown expiry) | +| `rise-github-package-registry` | Repository deploy token ([wheel_builder](https://gitlab.com/riseproject/python/wheel_builder/-/settings/repository#js-deploy-tokens)) | n/a (deploy token) | `read_package_registry`, `write_package_registry` | Push wheels GitHub → GitLab package registry | 1 year (June 29th, 2027) |