From 6cff6542e2bbc0d69f03951174dd3b3ebd8fb82a Mon Sep 17 00:00:00 2001 From: Mats Wichmann Date: Sun, 31 Jul 2022 08:49:09 -0600 Subject: Split requirements files Update CI and other files that refer to them. While we're at it, switch to choco pkg winflexbison3 from winflexbison. Update RELEASE file and template as well, to eliminate possible confusion (not that any has been reported). Signed-off-by: Mats Wichmann --- .appveyor/install.bat | 4 +- .github/workflows/scons-package.yml | 4 +- .travis.yml | 2 +- CONTRIBUTING.rst | 74 +++++++++++++++++++++++++------------ RELEASE.txt | 15 ++++++-- pyproject.toml | 13 +++++++ requirements-dev.txt | 12 ++++++ requirements-pkg.txt | 13 +++++++ requirements.txt | 20 +--------- template/RELEASE.txt | 15 ++++++-- 10 files changed, 117 insertions(+), 55 deletions(-) create mode 100644 pyproject.toml create mode 100644 requirements-dev.txt create mode 100644 requirements-pkg.txt diff --git a/.appveyor/install.bat b/.appveyor/install.bat index d009e54..b014dc7 100644 --- a/.appveyor/install.bat +++ b/.appveyor/install.bat @@ -5,8 +5,8 @@ set PATH=C:\\%WINPYTHON%;C:\\%WINPYTHON%\\Scripts;C:\\ProgramData\\chocolatey\\b C:\\%WINPYTHON%\\python.exe -m pip install -U --progress-bar off pip setuptools wheel C:\\%WINPYTHON%\\python.exe -m pip install -U --progress-bar off coverage codecov set STATIC_DEPS=true & C:\\%WINPYTHON%\\python.exe -m pip install -U --progress-bar off lxml -C:\\%WINPYTHON%\\python.exe -m pip install -U --progress-bar off -r requirements.txt +C:\\%WINPYTHON%\\python.exe -m pip install -U --progress-bar off -r requirements-dev.txt REM install 3rd party tools to test with -choco install --allow-empty-checksums dmd ldc swig vswhere xsltproc winflexbison +choco install --allow-empty-checksums dmd ldc swig vswhere xsltproc winflexbison3 set SCONS_CACHE_MSVC_CONFIG=true set diff --git a/.github/workflows/scons-package.yml b/.github/workflows/scons-package.yml index b124592..74eece6 100644 --- a/.github/workflows/scons-package.yml +++ b/.github/workflows/scons-package.yml @@ -23,10 +23,10 @@ jobs: run: | python -m pip install --upgrade pip setuptools wheel #python -m pip install flake8 pytest - if [ -f requirements.txt ]; then pip install -r requirements.txt; fi + if [ -f requirements-pkg.txt ]; then pip install -r requirements-pkg.txt; elif [ -f requirements.txt ]; then pip install -r requirements.txt; else; fi sudo apt-get update sudo apt-get -y install docbook-xml docbook-xsl xsltproc fop docbook-xsl-doc-pdf - # try to keeo the texlive install as small as we can to save some time/space + # try to keep the texlive install as small as we can to save some time/space sudo apt-get -y --no-install-recommends install texlive biber texmaker ghostscript texlive-latex-base texlive-latex-extra texlive-bibtex-extra texlive-font-utils latexmk # This is disabled until the run can be configured to only # check the code that matters, else we fail on non-essentials diff --git a/.travis.yml b/.travis.yml index 048fb3c..bae69bc 100644 --- a/.travis.yml +++ b/.travis.yml @@ -15,7 +15,7 @@ os: install: # needed for Docbook tests, must be in virtualenv context - - pip install -r requirements.txt + - pip install -r requirements-dev.txt # do the rest of the image setup - ./.travis/install.sh diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst index d4df14a..9d6e68c 100644 --- a/CONTRIBUTING.rst +++ b/CONTRIBUTING.rst @@ -2,7 +2,7 @@ Contributing to SCons ##################### Introduction -=========== +============ Thanks for taking the time to contribute to SCons! @@ -106,7 +106,7 @@ Documentation is written in a markup language which is a light extension of Docbook-XML, and the doc build consists of translating to pure docbook, then using standard tools to generate HTML and PDF outputs from that. There's lots more -on the documentation process at the Documentation Toolchain page:: +on the documentation process at the Documentation Toolchain page: https://github.com/SCons/scons/blob/master/doc/overview.rst @@ -132,7 +132,7 @@ can do that with the ``C`` (change directory) option:: There is another approach that kind of reverses that order: construct a Python virtualenv and install the development tree in it. -If you're not familiar with virtualenvs, there's an example here:: +If you're not familiar with virtualenvs, there's an example here: https://scons-cookbook.readthedocs.io/en/latest/#setting-up-a-python-virtualenv-for-scons @@ -172,7 +172,22 @@ of Python. The tools which will be used to to actually construct the project, such as compilers, documentation production tools, etc. should of course be installed by the appropriate means. In order to develop SCons and run its test suite, there are some dependencies, -listed in the ``requirements.txt`` file. +listed in the ``requirements-dev.txt`` file. Install these with:: + + $ python -m pip install -r requirements-dev.txt + +For building the SCons packages and documentation there are some further +requirements, you can get these with:: + + $ python -m pip install -r requirements-pkg.txt + +The requirements are inclusive so you only need the latter to get +everything installed. + +There are other, non-Python requirements to do a doc build. These +are system-specific. See bin/scons_dev_master.py for the set up that +works for Ubuntu systems. + Making Changes ============== @@ -180,15 +195,15 @@ Making Changes Virtually all of the SCons functionality exists in the "build engine," the ``SCons`` subdirectory hierarchy that contains all of the modules that make up SCons. The ``scripts/scons.py`` wrapper script exists mainly to find -the appropriate build engine library and then execute it. +the appropriate build engine module and execute it. In order to make your own changes locally and test them by hand, simply edit modules in the local ``SCons`` subdirectory tree and then run -(see the section above about `Executing SCons Without Installing`_):: +(see the section `Executing SCons Without Installing`_):: $ python scripts/scons.py [arguments] -(or, if using the virtualenv/editable approach, ``scons [arguents]``) +Or, if using the virtualenv/editable approach: ``scons [arguments]`` Note that the regular SCons development process makes heavy use of automated testing. See the `Testing`_ and `Typical Development Workflow`_ sections below for more @@ -320,7 +335,8 @@ subdirectory, and do not require that a build be performed first. Typical Development Workflow ============================ - Caveat: The point of this section isn't to describe one dogmatic workflow. +.. hint:: + The point of this section is not to describe one dogmatic workflow. Just running the test suite can be time-consuming, and getting a patch to pass all of the tests can be more so. If you're genuinely blocked, it may make more sense to submit a patch with a note about which tests still @@ -357,7 +373,9 @@ platform, Windows users can translate as appropriate)): $ python runtest.py -a -o test.log - Be patient, there are more than 1100 test scripts in the whole suite! + Be patient, there are more than 1100 test scripts in the whole suite + (using a ``-j`` option pretty much always helps. For example, if you have + an 8-core processor, try ``runtest.py -j 8```). If any test scripts fail, they will be listed in a summary at the end of the log file. Some test scripts may also report NO RESULT because (for example) @@ -392,11 +410,21 @@ example, if you're making a change to just the Java support, you might start looking for regressions by just running the ``test/Java/\*.py`` tests instead of running all tests with ``runtest.py -a``. +- To actually submit the fix and any test work as a Pull Request, + there will be some version control steps. For example:: + + $ git checkout -b fix-1387 + $ git modified # check that this reports your expected list + $ git add `git modified` + $ git commit -s # fill in a good description of your changes + + And proceed to push the change as a PR. + Building Packages ================= -We use SCons (version 3.1.2 or later) to build its own packages. If you +We use SCons (since version 3.1.2) to build its own packages. If you already have an appropriate version of SCons installed on your system, you can build everything by simply running it:: @@ -411,12 +439,12 @@ you can run the build directly from the source tree Those are full builds: depending on the utilities installed on your system, any or all of the following packages will be built:: - SCons-4.3.0-py3-none-any.whl - SCons-4.3.0ayyyymmdd.tar.gz - SCons-4.3.0ayyyymmdd.zip - scons-doc-4.3.0ayyyymmdd.tar.gz - scons-local-4.3.0ayyyymmdd.tar.gz - scons-local-4.3.0ayyyymmdd.zip + SCons-4.4.0-py3-none-any.whl + SCons-4.4.0ayyyymmdd.tar.gz + SCons-4.4.0ayyyymmdd.zip + scons-doc-4.4.0ayyyymmdd.tar.gz + scons-local-4.4.0ayyyymmdd.tar.gz + scons-local-4.4.0ayyyymmdd.zip The ``SConstruct`` file is supposed to be smart enough to avoid trying to build packages for which you don't have the proper utilities installed. @@ -465,14 +493,14 @@ bootstrap.py Obsolete packaging logic - ignore this. debian/ - Files needed to construct a Debian package. The contents of this directory - are dictated by the + Files needed to construct a Debian package. + The contents of this directory are dictated by the `Debian Policy Manual `). The package will not be accepted into the Debian distribution unless the contents of this directory satisfy the relevant Debian policies. - At this point, this is a sample; SCons is packaged in the Debian - project (and thus inherited by projects which derive from it, if - they haven't made their own packages). See: + At this point, this is a sample; SCons is packaged for Debian by the + Debian project itself (and thus inherited by projects which derive from it, + if they haven't made their own packages). See: - `Debian scons packages `_ - `Ubuntu scons packages `_ @@ -528,8 +556,8 @@ SCons/ documentation stubs kept together with pieces of the engine. test/ - End-to-end tests of the SCons utility itself. These are separate from the - individual module unit tests. + End-to-end tests of the SCons utility itself. + These are separate from the individual module unit tests. testing/ SCons testing framework. diff --git a/RELEASE.txt b/RELEASE.txt index 356691d..439c863 100755 --- a/RELEASE.txt +++ b/RELEASE.txt @@ -1,10 +1,17 @@ -A new SCons release, 4.4.1, is now available -on the SCons download page: +If you are reading this in the git repository, the contents +refer to *unreleased* changes since the last SCons release. +Past official release announcements appear at: - https://scons.org/pages/download.html + https://scons.org/tag/releases.html +================================================================== -Here is a summary of the changes since 4.4.1: +A new SCons release, 4.4.1, is now available on the SCons download page: + + https://scons.org/pages/download.html + + +Here is a summary of the changes since 4.4.0: NEW FUNCTIONALITY ----------------- diff --git a/pyproject.toml b/pyproject.toml new file mode 100644 index 0000000..1725456 --- /dev/null +++ b/pyproject.toml @@ -0,0 +1,13 @@ +[build-system] +build-backend = "setuptools.build_meta" +requires = ["setuptools", "wheel"] + +# for black and mypy, set the lowest Python version supported +[tool.black] +quiet = true +target-version = ['py36'] +skip-string-normalization = true + +[mypy] +python_version = 3.6 + diff --git a/requirements-dev.txt b/requirements-dev.txt new file mode 100644 index 0000000..6f9855f --- /dev/null +++ b/requirements-dev.txt @@ -0,0 +1,12 @@ +# Dependencies for development - mainly for running tests + +-r requirements.txt + +# for now keep pinning "known working" lxml, +# it's been a troublesome component in the past. +lxml==4.9.1 +ninja + +# Needed for test/Parallel/failed-build/failed-build.py +# Also for test/ninja/shutdown_scons_daemon.py +psutil diff --git a/requirements-pkg.txt b/requirements-pkg.txt new file mode 100644 index 0000000..3c2c385 --- /dev/null +++ b/requirements-pkg.txt @@ -0,0 +1,13 @@ +# Dependencies for packaging, and distribution, including the docs build + +-r requirements-dev.txt + +# Check if the README.rst will render o.k. on pypi and not block an upload +# Can be used with twinecheck +# See: https://github.com/pypa/readme_renderer +readme-renderer + +# sphinx pinned because it has broken several times on new releases +sphinx>=5.1.1 +sphinx_rtd_theme +rst2pdf diff --git a/requirements.txt b/requirements.txt index 0449bc8..e2aa5ca 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,19 +1 @@ -# Packages needed for development, packaging, and distribution, but not for running SCons - -# This will check if the README.rst will render o.k. on pypi and not block an upload -# Can be used with twinecheck -# See: https://github.com/pypa/readme_renderer -readme-renderer -#sphinx<=5.0.0 -sphinx>=5.1.1 -sphinx_rtd_theme -rst2pdf -# for now keep pinning "known working" lxml, -# it's been a troublesome component in the past. -lxml==4.9.1 -rst2pdf -ninja - -# Needed for test/Parallel/failed-build/failed-build.py -# Also for test/ninja/shutdown_scons_daemon.py -psutil +# No dependencies for running SCons diff --git a/template/RELEASE.txt b/template/RELEASE.txt index 89d634c..439c863 100755 --- a/template/RELEASE.txt +++ b/template/RELEASE.txt @@ -1,10 +1,17 @@ -A new SCons release, 4.1.0, is now available -on the SCons download page: +If you are reading this in the git repository, the contents +refer to *unreleased* changes since the last SCons release. +Past official release announcements appear at: - https://scons.org/pages/download.html + https://scons.org/tag/releases.html +================================================================== -Here is a summary of the changes since 4.1.0: +A new SCons release, 4.4.1, is now available on the SCons download page: + + https://scons.org/pages/download.html + + +Here is a summary of the changes since 4.4.0: NEW FUNCTIONALITY ----------------- -- cgit v0.12 From e1fe899982a574108ad6a4a824390f1c0e98a108 Mon Sep 17 00:00:00 2001 From: Mats Wichmann Date: Thu, 4 Aug 2022 12:11:04 -0600 Subject: Fix error in github setup Signed-off-by: Mats Wichmann --- .github/workflows/scons-package.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/scons-package.yml b/.github/workflows/scons-package.yml index 74eece6..91acecb 100644 --- a/.github/workflows/scons-package.yml +++ b/.github/workflows/scons-package.yml @@ -23,7 +23,7 @@ jobs: run: | python -m pip install --upgrade pip setuptools wheel #python -m pip install flake8 pytest - if [ -f requirements-pkg.txt ]; then pip install -r requirements-pkg.txt; elif [ -f requirements.txt ]; then pip install -r requirements.txt; else; fi + if [ -f requirements-pkg.txt ]; then pip install -r requirements-pkg.txt; elif [ -f requirements.txt ]; then pip install -r requirements.txt; fi sudo apt-get update sudo apt-get -y install docbook-xml docbook-xsl xsltproc fop docbook-xsl-doc-pdf # try to keep the texlive install as small as we can to save some time/space -- cgit v0.12 From c787ec1ba745e8b52d853004a891c722bed17077 Mon Sep 17 00:00:00 2001 From: Mats Wichmann Date: Fri, 5 Aug 2022 06:37:33 -0600 Subject: Change CONTRIBUTING from review [ci skip] Signed-off-by: Mats Wichmann --- CONTRIBUTING.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst index 9d6e68c..6fc99e9 100644 --- a/CONTRIBUTING.rst +++ b/CONTRIBUTING.rst @@ -424,7 +424,7 @@ running all tests with ``runtest.py -a``. Building Packages ================= -We use SCons (since version 3.1.2) to build its own packages. If you +We use SCons (version 3.1.2 or newer) to build its own packages. If you already have an appropriate version of SCons installed on your system, you can build everything by simply running it:: -- cgit v0.12