Tox 4.0.0 has some incompatible changes, epecially more
strict on allowlist_externals. Tempest recently changed
allowlist_externals not to be *[1] causing the failure
on jobs where lib/tempest failing to run the tempest
as command in virtual env.

----------
venv: commands[0]> tempest verify-config -uro /tmp/tmp.qH5KgJHTF4
venv: failed with tempest is not allowed, use allowlist_externals to allow it
------

We do not need to test/fix the <=stable/zed branches with tox 4.0.0
and pinning them with the compatible tox version of the time stable
brnaches were releaased is better way.

This commit proposes:
1. Pinning the tox<4.0.0 for <=stable/ze branches testing
2. Workaround to unblock the master gate by pinning it <4.0.0 but
we should make our testing compatible with tox 4.0.0 soon.

Related-Bug: #1999183

[1] https://review.opendev.org/c/openstack/tempest/+/865314 devstack based job started failing to run tempest command on venv.

Change-Id: I9a138af94dedc0d8ce5a0d519d75779415d3c30b
(cherry picked from commit ba54baa4253f2c770760b59a4a8f28317e5e260d)
(cherry picked from commit a3227ba0c0a6b951b301e556f78e634e6ce9e770)
(cherry picked from commit 395f44708506b30b29b35dd655dddd434bac9a08)
(cherry picked from commit 02f286a80a478d662e079fe066d311e4cf7264d8)
(cherry picked from commit 0a72476d895d925fdd37b4e42e2930b415b99d8f)
(cherry picked from commit 02b6981f)

In case of online mode, there is a procedure to recreate tempest venv.
For consistency of tempest venv during the entire stack.sh process,
add logic to consider the TEMPEST_VENV_UPPER_CONSTRAINTS option here.

Closes-bug: #1980483
Signed-off-by: June Yi <june.yi@samsung.com>
Change-Id: I0cea282152fd363af8671cab1b5f733ebe2bd4df
(cherry picked from commit 8355e8130652da0a6ed87295998200d0bcb9a58e)

Without it segment plugin fails to connect with
placement api. Configure the placement section
if service is deployed.

Closes-Bug: #1973783
Change-Id: Ie7f37770a04f622735cf2263c601257669ab5064
(cherry picked from commit 92a34dbe)
(cherry picked from commit ebd72a5e)

Stable ussuri is in Extended maintenance state[1] and
we need to pin tempest in stable/ussuri testing.

We use Tempest 26.1.0 because of oslo.utils version bump in Tempest
27.0.0 and 28.0.0 which is not compatible with stable/ussuri
upper constraints so these versions do not work in stable/ussuri.
Tempest 26.1.0 version is compatible for ussuri testing from tempest
side as well as from upper constraints side.

To merge this, making openstacksdk-functional-devstack job
as non-voting util we fix- https://storyboard.openstack.org/#!/story/2010057

Depends-On: https://review.opendev.org/c/openstack/tempest/+/843045

Change-Id: I5a637bdcdf61e71c7e948187f686de40f5ef0dca

git commit [1] introduced a new behaviour to work around a CVE that
disallows any git operations in directories not owned by the current
user.

This may seem unrelated to installation, but it plays havoc with PBR,
which calls out to git to get to get revision history.  So if you are
"pip install"-ing from a source tree you don't own, the PBR git calls
in that tree now fail and the install blows up.

This plays havoc with our model.  Firstly, we checkout all code as
"stack" then install it globally with "sudo" (i.e. root) -- which
breaks.  We also have cases of essentially the opposite -- checkouts
we have installed as root, but then run tox in them as a regular user;
tox wants to install the source in its venv but now we have another
user conflict.

This uses the only available configuration option to avoid that by
globally setting the source directories we clone as safe.  This is an
encroachment of the global system for sure, but is about the only
switch available at the moment.  For discussion of other approaches,
see [2].

Also, squashing the below backport which is needed for bionic
- https://review.opendev.org/q/I941ef5ea90970a0901236afe81c551aaf24ac1d8

Related-Bug: https://bugs.launchpad.net/devstack/+bug/1968798

[1] https://github.com/git/git/commit/8959555cee7ec045958f9b6dd62e541affb7e7d9
[2] https://review.opendev.org/c/openstack/devstack/+/837636

Change-Id: Ib9896a99b6d6c4d359ee412743ce30512b3c4fb7
(cherry picked from commit 676dcaf9)

CentOS8 is going to be EOL soon and opendev
is planning to drop the centOS8 image from CI
- http://lists.openstack.org/pipermail/openstack-discuss/2022-January/026621.html

Depends-On: https://review.opendev.org/c/openstack/python-cinderclient/+/826092
Change-Id: Ife528fa6383b14a89b5d4a330ae60c2f82116798

We have made the grenade job non-voting in the check queue, but it is
stilled pulled in as voting in gate via the integrated-gate-py3
template. Stop using that template and instead reference the
tempest-full-py3 job directly in check and gate.

Change-Id: I02e3208df53e7744c4be9c62172b78f08da0514e

CentOS8 is going to be EOL soon and opendev
is planning to drop the centOS8 image from CI
- http://lists.openstack.org/pipermail/openstack-discuss/2022-January/026621.html

Devstack stable/wallaby and onwards has replaced the
centOS testing with centOS Stream. For older stable branches
(<= stable/victoria) we can either replace it with centOS Steam
or do the removal of CentOS8 and adding CentOS Stream testing
in two steps 1. drop the centOS8 testing as it cannot
be supported in CI 2. add centOS Stream job if possible.

This commit propose the step 1 as required by opendev image removal.

[Stable only]

Change-Id: I36751569d92fbc5084b8308d423a75318ae7d406
(cherry picked from commit 3602e434)

Since pip v22, python3.6 is not supported (the minimum version is
python3.7). This patch adds the reference for the pip3.6 URL to be
used instead of the default one.

Conflicts:
  tools/install_pip.sh

Depends-On: https://review.opendev.org/c/openstack/devstack/+/827421
Closes-Bug: #1959600
Change-Id: Iab2c391d5388461fe9e9037cee81884ce8032e72
(cherry picked from commit a756f4b9)
(cherry picked from commit 13da39fc)
(cherry picked from commit a4369c8b)
(cherry picked from commit 5f5d9000a7bf3e1b4aee78a06816c43588fa03b9)

The current initiator name embedded in our CI images is not unique at
present and can often cause failures during live migrations with
attached volumes. This change ensures the name is unique by running
iscsi-iname again and overwriting the existing name.

We could potentially do this during the image build process itself but
given that devstack systems are not supposed to be multi-purpose this
should be safe to do during the devstack run.

NOTE(lyarwood): Conflict due to
If2f74f146a166b9721540aaf3f1f9fce3030525c not being present on
stable/wallaby.

Conflicts:
    lib/nova

Closes-Bug: #1945983
Change-Id: I9ed26a17858df96c04be9ae52bf2e33e023869a5
(cherry picked from commit 714826d1)
(cherry picked from commit ee629cc7)
(cherry picked from commit a41fff99)
(cherry picked from commit 43364b71)

rdo-release.el8.rpm rpm points to latest RDO release,
so use it for master, for stable releases use corresponding
release rpm.

Change-Id: I508eceb00d7501ffcfac73d7bc2272badb241494
(cherry picked from commit 0456baae)
(cherry picked from commit 3e54b3c0)

from Tempest to DevStack as it tests DevStack side of things and
is useful for projects not using Tempest.

Verbatim copy except for the devstack- prefix and the /devstack/
path.

Change-Id: Ie166730843f874b9c99e37244e460d7ad33b7eeb
(cherry picked from commit 2fb8c7a5)

Ussuri is python3-only.

Change-Id: Ie002faf4c96ac7f207207a481c057b8df0289e6c
(cherry picked from commit 12484c4c)

devstack unit test job does not set any nodeset
and so does use default nodeset defined in base jobs
in opendev. When opendev switches the default nodeset to
the latest distro version, devstack unit test job can start
failing. Example:

- https://review.opendev.org/q/I01408f2f2959b0788fe712ac268a526502226ee9
- https://review.opendev.org/q/Ib1ea47bc7384e1f579cb08c779a32151fccd6845

To avoid such a situation in future, let's set the working nodeset
for this job also so that when we cut the stable branch we can
run it on the working distro version. 

Change-Id: I302140778fedf08bc8ba72c453837fa7b8f8f9ae

Sphinx 4.0.0 added a new dependency [0] which is causing the job to fail
at the moment.

This patch fix the problem by adding UC to the docs jobs.

[0] https://www.sphinx-doc.org/en/master/changes.html

 (LaTeX: add
tex-gyre font dependency)

Change-Id: I28019331017405c06577ada88f8e9f6d9a2afc23
Signed-off-by: Lucas Alvares Gomes <lucasagomes@gmail.com>
(cherry picked from commit 69a66fb6)

Due to issue on stckviz side, job start
failing with POST_FAILURE. If we fix the
issue still we need to wait for periodic job
periodic-package-stackviz-element to publish
the latest tarball on https://tarballs.openstack.org/stackviz/dist/.

Let's not fail the job for any issue occur during
stackviz processing.

Closes-Bug: 1863161

Change-Id: Ifee04f28ecee52e74803f1623aba5cfe5ee5ec90
(cherry picked from commit 580fec54)
(cherry picked from commit 800eb4dd)
(cherry picked from commit 3b2feba2)

CentOS 8.3 changed the name of the PowerTools repository to powertools:
https://wiki.centos.org/Manuals/ReleaseNotes/CentOS8.2011#Yum_repo_file_and_repoid_changes

With this repository disabled, DevStack fails to install libyaml-devel,
which causes a failure to install many packages. In my environment
DevStack stopped with an error caused by a missing wget.

Keep the command using the old repository name, for compatibility with
older CentOS releases.

Change-Id: I5541a8aee8467abf10ce8a10d770618bdd693f02
(cherry picked from commit f3611227)

We use Tempest master for testing the supported stable
branches so using master upper constraints works fine but
when we need to use old Tempest in the below cases then master
upper constraints do not work and devstack will not be
able to install Tempest in vnenv:

- Testing Extended Maintenance branch
- Testing py2.7 jobs until stable/train with in-tree tempest plugins

This commit adds a variable to set the compatible upper constraint
to use for Tempest's old version.

Few of the current failure which can be fixed by this new configurable var:
- networking-generic-switch-tempest-dlm-python2
  - https://zuul.opendev.org/t/openstack/build/ebcf3d68d62c4af3a43a222aa9ce5556
- devstack-platform-xenial on stable/steinand stable/train
  - https://zuul.opendev.org/t/openstack/build/37ffc1af6f3f4b44b5ca8cbfa27068ac

Change-Id: I5b2217d85e6871ca3f7a3f6f859fdce9a50d3946
(cherry picked from commit 3bdc8f66)
(cherry picked from commit 2e4b708e)

As added in notes for INSTALL_TEMPEST variable we need to set
this as False for stable branch so that devstack does not
install Tempest at system wide.

- https://github.com/openstack/devstack/blob/aa2821eb89ab9c8048509d15fe41215c163d2f50/lib/tempest#L61

This should be done at the time when we cut the stable branch but
we forgot to do that for ussuri and victoria.

Change-Id: I23c77f98c2e969d8046d5212b883e343c36cd1b1
(cherry picked from commit 9a8c0623)

Tempest option scenario.img_dir has been deprecated more than 4 years, it's
time to remove it from devstack.
img_file option should contain the full path to the image.
This patch removes setting of img_dir option and makes img_file
one contain the full path of an image.

Change-Id: I71102095f3603915f0bc7d21f2e18c4eac4e95ec
Related-Bug: #1393881

available_features for Neutron tempest tests is default to "all" and
used from Victoria to enable/disable tests that has no API extension.
It is added here to make sure that stable gating will not execute these
tests (like for IPv6 metadata in neutron-tempest-plugin or updating
min_bw QoS values for port already attached to VMs)

Cherry-picked from stable/victoria patch
(https://review.opendev.org/c/openstack/devstack/+/769885), but the list
for available_features is empty on Ussuri.

Change-Id: Ic6056218e51e6b0da554bcdf139214d2229ba3c1
Related-Bug: #1882804
(cherry picked from commit e34a8720)

Since the introduction of I8f24b839bf42e2fb9803dc7df3a30ae20cf264
s-proxy is no longer able to launch as keystonemiddleware (listed under
test-requirements.txt) has not been installed.

keystonemiddleware is listed as extras requirements in swift
- https://github.com/openstack/swift/blob/e0d46d77fa740768f1dd5b989a63be85ff1fec20/setup.cfg#L79

Let's install swift keystone extra requirements also.

Closes-Bug: #1909018
Change-Id: I02c692e95d70017eea03d82d75ae6c5e87bde8b1
(cherry picked from commit 04b0b615)
(cherry picked from commit 90651cb1)

This patch caps pip version during bootstrap to avoid the issue:

"ERROR: Links are not allowed as constraints"

A proper fix would be to adapt to new pip behavior.


Change-Id: I1feed4573820436f91f8f654cc189fa3a21956fd

While RHEL 7 is detected as RedHatEnterpriseServer, RHEL 8 is
RedHatEnterprise.

  $ lsb_release -i -s
  RedHatEnterprise

Change-Id: I3d750d808c6ebea9c230f0508cdbc016415b9922
(cherry picked from commit 587e0a35)

fedora-latest now points at Fedora 32 which is not supported prior to
Victoria. As we don't support earlier releases of Fedora outside their
own life cycles we should just remove Fedora jobs from these older
branches.

Change-Id: I9b1e674f50f2966d46bced6663f4a2735be0685d

Nova has newer qemu requirements which needs opensuse 15.2 to meet. The
infra opensuse 15 images are 15.2 now to accomodate that. Update
opensuse's supported version to match.

Change-Id: I6f3c5234920b185b2b0cd9c358371402f7a7b922
(cherry picked from commit 471f1625)

This is will allow the openstack/nova project to facilitate a minimum
required version bump of QEMU and libvirt within the libvirt virt driver
in I8e349849db0b1a540d295c903f1470917b82fd97 ahead of the planned switch
to focal later in Victoria.

NOTE(lyarwood): This also has to be backported to stable/ussuri to allow
grenade to pass on master. Otherwise we don't install the required UCA
provided versions of QEMU and libvirt during the initial deploy, cauing
n-cpu to fail to start after the upgrade.

Change-Id: I85eb45632ff229676f7c29708f4a7cc64b3d90e3
(cherry picked from commit 7f7f488b)

This should result in the fedora-latest nodeset from master being used
instead, allowing us to move that nodeset to fedora-32.

Change-Id: I6a66d85b1fc5a4c3f5fa651d33ff08260a87574f

Tempest xenapi related tests are skipped for Victoria
onwards
- https://review.opendev.org/#/c/749527/

We need to keep runnnig those test till stable ussuri.

Depends-On: https://review.opendev.org/#/c/749527/
Change-Id: If67186e46cb3c4f4505d006ec6389057925ce44c

A new setuptools release has changed the way pip installs are done,
see [0]. With this change we switch back to using the distro
method for global pip installs.

Temporarily make grenade jobs non-voting in order to allow this
patch to be backported.

[0] http://lists.openstack.org/pipermail/openstack-discuss/2020-August/016905.html

Change-Id: I5d8aa0e58e0409c54451b51de5eb70ba9a68d849
(cherry picked from commit 18b4251b)