自己写一个包¶
本文基于我写的包:
npnn
,参考Packaging-Projects
包的代码结构¶
npnn
在GitHub上的项目结构是:
npnn
├── LICENSE
├── dataset.py
├── model.py
├── pyproject.toml
├── readme.md
├── requirements.txt
├── search.py
├── search_result.csv
├── src
│ └── npnn
│ ├── __init__.py
│ ├── autograd.py
│ ├── base.py
│ ├── functional.py
│ ├── nn.py
│ └── optim.py
├── test.py
├── tests
│ └── test_autograd.py
├── train.py
├── utils.py
└── viz.py
npnn
├── LICENSE # 许可文件
├── pyproject.toml # Python项目配置文件
├── readme.md # readme
├── src # Python项目源代码
│ └── npnn # module
│ ├── __init__.py # module初始化方法
│ ├── autograd.py # 反向传播算法
│ ├── base.py # 基类
│ ├── functional.py # 函数操作
│ ├── nn.py # 神经网络
│ └── optim.py # 优化器
├── tests # pytest测试
│ └── test_autograd.py # 测试具体内容
pyproject.toml¶
这个文件是Python项目的核心文件,描述了Python项目的核心信息。
主要包括以下板块:
构建系统¶
我选择了setuptools
[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"
项目核心信息¶
包括:
- 名称:name
- 版本号:version,我的习惯是
va.b.c
,a是大版本,b是小版本,c是小更新。 - 依赖:dependencies
- Python要求
- 作者信息:authors
- 项目描述:description
- 说明文档:readme
- 许可:license
- 关键词:keywords
- 分类:classifiers
[project]
name = "npnn"
version = "0.1.1"
dependencies = [
"numpy"
]
requires-python = ">=3.9"
authors = [
{ name="Yang Zhang", email="[email protected]" },
]
description = "NumPy Neural Network"
readme = "readme.md"
license = {file = "LICENSE"}
keywords = ["numpy", "neural network", "machine learning", "autograd"]
classifiers = [
"Development Status :: 3 - Alpha",
"Intended Audience :: Developers",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
]
可选依赖¶
[project.optional-dependencies]
cuda11x = ["cupy-cuda11x"]
cuda12x = ["cupy-cuda12x"]
python -m pip install npnn[cuda12x]
cupy-cuda12x
。 项目主页¶
[project.urls]
Homepage = "https://github.com/AIboy996/npnn"
测试¶
使用pytest
进行测试:
python -m pip install -U pytest
$ python -m pytest
---> 100%
===== test session starts ==============================
platform darwin -- Python 3.11.8, pytest-8.1.1, pluggy-1.4.0
rootdir: /Users/yang/Desktop/Project/npnn
configfile: pyproject.toml
plugins: anyio-3.7.1
collected 6 items
tests/test_autograd.py ...... [100%]
==== 6 passed in 0.58s ===============================
构建¶
使用build
来构建Python项目:
python -m pip install -U build
python -m build
./dist
生成构建好的软件包: dist
├── npnn-0.1.1-py3-none-any.whl
└── npnn-0.1.1.tar.gz
发布¶
需要提前在PyPI注册账号、获取token
使用twine
来发布Python项目:
python -m pip install -U twine
python -m twine upload -u __token__ -p PYPI_API_TOKEN --skip-existing dist/*
自动化workflow¶
使用GitHub提供的actions功能可以实现测试、构建、发布的自动化。
.github/workflows/pythoh-package-publish.yml
# test and publish Python Package to pypi
name: pytest
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
jobs:
test:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ["3.9", "3.10", "3.11", "3.12"]
steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install flake8 pytest
python -m pip install -e .
python -m pip install torch --index-url https://download.pytorch.org/whl/cpu
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Lint with flake8
run: |
# stop the test if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Test with pytest
run: |
pytest
publish:
needs: test
runs-on: ubuntu-latest
environment: pypi_publish
permissions:
contents: read
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.10'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install build twine
- name: Build package
run: python -m build
- name: Publish package
run: python -m twine upload -u __token__ -p ${{ secrets.PYPI_API_TOKEN }} --skip-existing dist/*
这个workflow文件分为两个jobs:
- test:分别在
Python 3.9 3.10 3.11 3.12
上安装依赖、运行测试。并且进行flake8的语法检查。 - publish:使用build构建,然后使用twine发布
并且publish是依赖于test的,如果后者没有正常运行结束,publish就不会运行。
值得一提的是高亮的两行配置,这里使用了GitHub的environments功能,用环境变量的方式存储了PyPI的token。
Last update: 2024-04-28 00:26:41
Created: 2023-01-18 21:12:55
Created: 2023-01-18 21:12:55