Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 7 additions & 27 deletions .github/workflows/_tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,32 +8,12 @@ on:
workflow_call:

jobs:
# Create a custom unit tests job, because apify-shared-python doesn't use codecov report.
unit_tests:
name: Unit tests
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest]
python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]
runs-on: ${{ matrix.os }}

steps:
- name: Checkout repository
uses: actions/checkout@v6

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v6
with:
python-version: ${{ matrix.python-version }}

- name: Set up uv package manager
uses: astral-sh/setup-uv@v7
with:
python-version: ${{ matrix.python-version }}

- name: Install Python dependencies
run: make install-dev

- name: Run unit tests
run: make unit-tests-cov
uses: apify/workflows/.github/workflows/python_unit_tests.yaml@main
with:
python-versions: '["3.10", "3.11", "3.12", "3.13", "3.14"]'
operating-systems: '["ubuntu-latest", "windows-latest"]'
# Apify shared don't use codecov, but we have to provide these inputs.
python-version-for-codecov: "3.14"
operating-system-for-codecov: ubuntu-latest
4 changes: 2 additions & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,12 @@ repos:
hooks:
- id: lint-check
name: Lint check
entry: make lint
entry: uv run poe lint
language: system
pass_filenames: false

- id: type-check
name: Type check
entry: make type-check
entry: uv run poe type-check
language: system
pass_filenames: false
47 changes: 31 additions & 16 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,20 +8,37 @@ For local development, it is required to have Python 3.10 (or a later version) i

We use [uv](https://docs.astral.sh/uv/) for project management. Install it and set up your IDE accordingly.

We use [Poe the Poet](https://poethepoet.natn.io/) as a task runner, similar to npm scripts in `package.json`.
All tasks are defined in `pyproject.toml` under `[tool.poe.tasks]` and can be run with `uv run poe <task>`.

### Available tasks

| Task | Description |
| ---- | ----------- |
| `install-dev` | Install development dependencies |
| `check-code` | Run lint, type-check, and unit-tests |
| `lint` | Run linter |
| `format` | Fix lint issues and format code |
| `type-check` | Run type checker |
| `unit-tests` | Run unit tests |
| `unit-tests-cov` | Run unit tests with coverage |
| `build` | Build package |
| `clean` | Remove build artifacts and clean caches |

## Dependencies

To install this package and its development dependencies, run:

```sh
make install-dev
uv run poe install-dev
```

## Code checking

To execute all code checking tools together, run:

```sh
make check-code
uv run poe check-code
```

### Linting
Expand All @@ -31,7 +48,7 @@ We utilize [ruff](https://docs.astral.sh/ruff/) for linting, which analyzes code
To run linting:

```sh
make lint
uv run poe lint
```

### Formatting
Expand All @@ -41,7 +58,7 @@ Our automated code formatting also leverages [ruff](https://docs.astral.sh/ruff/
To run formatting:

```sh
make format
uv run poe format
```

### Type checking
Expand All @@ -51,30 +68,28 @@ Type checking is handled by [ty](https://docs.astral.sh/ty/), verifying code aga
To run type checking:

```sh
make type-check
uv run poe type-check
```

### Unit tests

We employ pytest as our testing framework, equipped with various plugins. Check pyproject.toml for configuration details and installed plugins.

We use [pytest](https://docs.pytest.org/) as a testing framework with many plugins. Check `pyproject.toml` for configuration details and installed plugins.

To run unit tests:

```sh
make unit-tests
uv run poe unit-tests
```

To run unit tests with HTML coverage report:

```sh
make unit-tests-cov
uv run poe unit-tests-cov
```

## Release process

Publishing new versions to [PyPI](https://pypi.org/project/apify) is automated through GitHub Actions.
Publishing new versions to [PyPI](https://pypi.org/project/apify-shared) is automated through GitHub Actions.

- **Beta releases**: On each commit to the master branch, a new beta release is automatically published. The version number is determined based on the latest release and conventional commits. The beta version suffix is incremented by 1 from the last beta release on PyPI.
- **Stable releases**: A stable version release may be created by triggering the `release` GitHub Actions workflow. The version number is determined based on the latest release and conventional commits (`auto` release type), or it may be overridden using the `custom` release type.
Expand All @@ -90,18 +105,18 @@ Publishing new versions to [PyPI](https://pypi.org/project/apify) is automated t

```toml
[project]
name = "apify"
name = "apify_shared"
version = "x.z.y"
```

4. Generate the distribution archives for the package:
4. Build the package:

```shell
uv build
```sh
uv run poe build
```

5. Set up the PyPI API token for authentication and upload the package to PyPI:
5. Upload to PyPI:

```shell
```sh
uv publish --token YOUR_API_TOKEN
```
36 changes: 0 additions & 36 deletions Makefile

This file was deleted.

19 changes: 19 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ dependencies = []
[dependency-groups]
dev = [
"dycw-pytest-only~=2.1.0",
"poethepoet<1.0.0",
"pre-commit~=4.3.0",
"pytest-asyncio~=1.1.0",
"pytest-cov~=6.2.0",
Expand Down Expand Up @@ -104,3 +105,21 @@ python-version = "3.10"

[tool.ty.src]
include = ["src", "tests", "scripts", "docs", "website"]

# Run tasks with: uv run poe <task>
[tool.poe.tasks]
clean = "rm -rf .coverage .pytest_cache .ruff_cache .ty_cache build dist htmlcov"
install-sync = "uv sync --all-extras"
install-dev = "uv sync --all-extras"
build = "uv build --verbose"
publish-to-pypi = "uv publish --verbose --token ${APIFY_PYPI_TOKEN_CRAWLEE}"
type-check = "uv run ty check"
unit-tests = "uv run pytest --numprocesses=auto --verbose --cov=src/apify_shared tests/unit"
unit-tests-cov = "uv run pytest --numprocesses=auto --verbose --cov=src/apify_shared --cov-report=html tests/unit"
check-code = ["lint", "type-check", "unit-tests"]

[tool.poe.tasks.lint]
shell = "uv run ruff format --check && uv run ruff check"

[tool.poe.tasks.format]
shell = "uv run ruff check --fix && uv run ruff format"
Loading
Loading