In train tempest is pinned to 26.1.0(requires paramkio>=2.7.0)
and since paramiko-3.0.0(requires cryptography>=3.3) release
jobs are failing as they are trying to compile cryptography
which needs rust to be installed or upgrade pip so
cryptography wheel binaries can be installed.

To setup rust ensure-rust role is executed in
pre run.

Also replace integrated-gate and integrated-gate-py3
templates with the required jobs in check and
gate queue as those templates have grenade jobs
in gate queue which are already marked as non-voting[1].

[1] https://review.opendev.org/c/openstack/devstack/+/838722

Partial-Bug: #2004490
Change-Id: I2ec1c34ec11f50fef06f71b4a01976aa1d92a6bb

In case of online mode, there is a procedure to recreate tempest venv.
For consistency of tempest venv during the entire stack.sh process,
add logic to consider the TEMPEST_VENV_UPPER_CONSTRAINTS option here.

Closes-bug: #1980483
Signed-off-by: June Yi <june.yi@samsung.com>
Change-Id: I0cea282152fd363af8671cab1b5f733ebe2bd4df
(cherry picked from commit 8355e8130652da0a6ed87295998200d0bcb9a58e)

Without it segment plugin fails to connect with
placement api. Configure the placement section
if service is deployed.

Closes-Bug: #1973783
Change-Id: Ie7f37770a04f622735cf2263c601257669ab5064
(cherry picked from commit 92a34dbe)
(cherry picked from commit ebd72a5e)

The stable/ussuri, stable/train, and stable/stein branches are
in Extended Maintenance mode and no longer released from, so make
neutron-grenade non-voting in check and remove it from the
gate.

All other grenade jobs are already non-voting in stable/train and
were removed from the gate four years ago by change I82f539bfa533.

Change-Id: I21975b270786687149b232866b2009c145ad9915
Related-bug: #1968798

git commit [1] introduced a new behaviour to work around a CVE that
disallows any git operations in directories not owned by the current
user.

This may seem unrelated to installation, but it plays havoc with PBR,
which calls out to git to get to get revision history.  So if you are
"pip install"-ing from a source tree you don't own, the PBR git calls
in that tree now fail and the install blows up.

This plays havoc with our model.  Firstly, we checkout all code as
"stack" then install it globally with "sudo" (i.e. root) -- which
breaks.  We also have cases of essentially the opposite -- checkouts
we have installed as root, but then run tox in them as a regular user;
tox wants to install the source in its venv but now we have another
user conflict.

This uses the only available configuration option to avoid that by
globally setting the source directories we clone as safe.  This is an
encroachment of the global system for sure, but is about the only
switch available at the moment.  For discussion of other approaches,
see [2].

Also, squashing the below backport which is needed for bionic
- https://review.opendev.org/q/I941ef5ea90970a0901236afe81c551aaf24ac1d8

Related-Bug: https://bugs.launchpad.net/devstack/+bug/1968798

[1] https://github.com/git/git/commit/8959555cee7ec045958f9b6dd62e541affb7e7d9
[2] https://review.opendev.org/c/openstack/devstack/+/837636

Change-Id: Ib9896a99b6d6c4d359ee412743ce30512b3c4fb7
(cherry picked from commit 676dcaf9)
(cherry picked from commit a9852abf)

Since pip v22, python3.6 is not supported (the minimum version is
python3.7). This patch adds the reference for the pip3.6 URL to be
used instead of the default one.

NOTE: stable/train had its own method to provide support for py3x.
This patch uses it instead of pushing the master patch approach.

This patch is also fixing the py35 pip URL string that should be
https://bootstrap.pypa.io/pip/3.5/get-pip.py.

Conflicts:
  tools/install_pip.sh

Closes-Bug: #1959600
Change-Id: Iab2c391d5388461fe9e9037cee81884ce8032e72

Stable train is in Extended maintenance state[1] and
we need to cap stable/train with older Tempest.

But because of oslo.utils version bump in Tempest 27.0.0
and 28.0.0 which is not compatible with stable/train
upper constraints so these versions do not work in stable/train

- https://zuul.opendev.org/t/openstack/build/1146c9ce9f474d0d82b99b48b61c531e

Let's use the Tempest 26.1.0 which is also compatible with stable/train
testing.

Related-Bug: #1955418
Change-Id: I376be5f5a4d693aeaaf74ca5285269f796103c15

This reverts commit 410d11eb.

Reason for revert: it caused the bug for tempest plugin, let's figure out that first and then cap the appropriate tempest version-  https://bugs.launchpad.net/tempest/+bug/1955418

Change-Id: Ie4305e7b6030d9edaa97a207a588b734cf7707dc

Stable train is in Extended maintenance state[1] and
As per Tempest stable branmch testing policy, it stopped
the support for stable/train in master version. Tempest
train-last (28.0.0) tag is last supported tag for stable/train.

Use that tag to test stable/train branch before any incompatible
change merge in Tempest master.

[1] https://releases.openstack.org/

Change-Id: I45d685725ec3204a7c0467085f87214f02ecbe1c

The current initiator name embedded in our CI images is not unique at
present and can often cause failures during live migrations with
attached volumes. This change ensures the name is unique by running
iscsi-iname again and overwriting the existing name.

We could potentially do this during the image build process itself but
given that devstack systems are not supposed to be multi-purpose this
should be safe to do during the devstack run.

NOTE(lyarwood): Conflict due to
If2f74f146a166b9721540aaf3f1f9fce3030525c not being present on
stable/wallaby.

Conflicts:
    lib/nova

Closes-Bug: #1945983
Change-Id: I9ed26a17858df96c04be9ae52bf2e33e023869a5
(cherry picked from commit 714826d1)
(cherry picked from commit ee629cc7)
(cherry picked from commit a41fff99)
(cherry picked from commit 43364b71)

from Tempest to DevStack as it tests DevStack side of things and
is useful for projects not using Tempest.

Verbatim copy except for the devstack- prefix and the /devstack/
path.

Change-Id: Ie166730843f874b9c99e37244e460d7ad33b7eeb
(cherry picked from commit 2fb8c7a5)

devstack unit test job does not set any nodeset
and so does use default nodeset defined in base jobs
in opendev. When opendev switches the default nodeset to
the latest distro version, devstack unit test job can start
failing. Example:

- https://review.opendev.org/q/I01408f2f2959b0788fe712ac268a526502226ee9
- https://review.opendev.org/q/Ib1ea47bc7384e1f579cb08c779a32151fccd6845

To avoid such a situation in future, let's set the working nodeset
for this job also so that when we cut the stable branch we can
run it on the working distro version. 

Change-Id: I302140778fedf08bc8ba72c453837fa7b8f8f9ae

Sphinx 4.0.0 added a new dependency [0] which is causing the job to fail
at the moment.

This patch fix the problem by adding UC to the docs jobs.

[0] https://www.sphinx-doc.org/en/master/changes.html

 (LaTeX: add
tex-gyre font dependency)

Change-Id: I28019331017405c06577ada88f8e9f6d9a2afc23
Signed-off-by: Lucas Alvares Gomes <lucasagomes@gmail.com>
(cherry picked from commit 69a66fb6)

Due to issue on stckviz side, job start
failing with POST_FAILURE. If we fix the
issue still we need to wait for periodic job
periodic-package-stackviz-element to publish
the latest tarball on https://tarballs.openstack.org/stackviz/dist/.

Let's not fail the job for any issue occur during
stackviz processing.

Closes-Bug: 1863161
Depends-On: https://review.opendev.org/c/openstack/grenade/+/785767

Change-Id: Ifee04f28ecee52e74803f1623aba5cfe5ee5ec90
(cherry picked from commit 580fec54)
(cherry picked from commit 800eb4dd)
(cherry picked from commit 3b2feba2)
(cherry picked from commit 4f9c1e08)

This has been failing to install with this message, which I think
means we need to change our URL for getting pip:

 The URL you are using to fetch this script has changed, and this one
 will no longer work. Please use get-pip.py from the following URL
 instead:
    https://bootstrap.pypa.io/pip/2.7/get-pip.py
 Sorry if this change causes any inconvenience for you!

Change-Id: Id62e91b1609db4b1d2fa425010bac1ce77e9fc51

We use Tempest master for testing the supported stable
branches so using master upper constraints works fine but
when we need to use old Tempest in the below cases then master
upper constraints do not work and devstack will not be
able to install Tempest in vnenv:

- Testing Extended Maintenance branch
- Testing py2.7 jobs until stable/train with in-tree tempest plugins

This commit adds a variable to set the compatible upper constraint
to use for Tempest's old version.

Few of the current failure which can be fixed by this new configurable var:
- networking-generic-switch-tempest-dlm-python2
  - https://zuul.opendev.org/t/openstack/build/ebcf3d68d62c4af3a43a222aa9ce5556
- devstack-platform-xenial on stable/steinand stable/train
  - https://zuul.opendev.org/t/openstack/build/37ffc1af6f3f4b44b5ca8cbfa27068ac

Change-Id: I5b2217d85e6871ca3f7a3f6f859fdce9a50d3946
(cherry picked from commit 3bdc8f66)
(cherry picked from commit 2e4b708e)
(cherry picked from commit 74a136ac)

get-pip.py is fixed for py2.7 in
https://review.opendev.org/c/openstack/devstack/+/772462

This commit support it for py3.5 as devstack stable/train and stein
still support ubuntu xenial which has py3.5

Also using Temepst 23.0.0 for xenial platform job which is
compatible version on py3.5.

Closes-Bug: #1913283
Change-Id: Icd40e41b5f3ef9a621ca1b9739875fcdd31abd74

get-pip.py dropped python 2.7 support, so we cannot use the default
get-pip.py for python 2.7 now. get-pip.py for python 2.7 is available
at https://bootstrap.pypa.io/2.7/get-pip.py.
This commit tries to use python 2.7 version of get-pip.py
for python 2.7 environment.

Depends-On: https://review.opendev.org/772486
Change-Id: I88ba370de8887a05c473fe7c19a61a319e8812d2
(cherry picked from commit 1382e933c5b2c16b276e8813fbc7cd9a1d6d4499)

available_features for Neutron tempest tests is default to "all" and
used from Victoria to enable/disable tests that has no API extension.
It is added here to make sure that stable gating will not execute these
tests (like for IPv6 metadata in neutron-tempest-plugin or updating
min_bw QoS values for port already attached to VMs)

Cherry-picked from stable/victoria patch
(https://review.opendev.org/c/openstack/devstack/+/769885), but the list
for available_features is empty on Ussuri and Train.

Change-Id: Ic6056218e51e6b0da554bcdf139214d2229ba3c1
Related-Bug: #1882804
(cherry picked from commit e34a8720)
(cherry picked from commit ef784e19)

Tempest option scenario.img_dir has been deprecated more than 4 years, it's
time to remove it from devstack.
img_file option should contain the full path to the image.
This patch removes setting of img_dir option and makes img_file
one contain the full path of an image.

Change-Id: I71102095f3603915f0bc7d21f2e18c4eac4e95ec
Related-Bug: #1393881

Since the introduction of I8f24b839bf42e2fb9803dc7df3a30ae20cf264
s-proxy is no longer able to launch as keystonemiddleware (listed under
test-requirements.txt) has not been installed.

keystonemiddleware is listed as extras requirements in swift
- https://github.com/openstack/swift/blob/e0d46d77fa740768f1dd5b989a63be85ff1fec20/setup.cfg#L79

Let's install swift keystone extra requirements also.

Closes-Bug: #1909018
Change-Id: I02c692e95d70017eea03d82d75ae6c5e87bde8b1
(cherry picked from commit 04b0b615)
(cherry picked from commit 90651cb1)
(cherry picked from commit 10fdf258)

While RHEL 7 is detected as RedHatEnterpriseServer, RHEL 8 is
RedHatEnterprise.

  $ lsb_release -i -s
  RedHatEnterprise

Change-Id: I3d750d808c6ebea9c230f0508cdbc016415b9922
(cherry picked from commit 587e0a35)

This is a test of installing openstack and then seeing if it works.
OpenStack components do not need test-requirements to operate,
that's why they are test-requirements.

Additionally, as we look forward to depsolver pip, this is going
to screw us because we don't apply constraints to linters, which
are expressed in - you guessed it, test-requirements.

NOTE(lyarwood): openstacksdk-functional is currently failing due to
I1e151454f39d51db52cdc7a23dbdcbf9b28c9381 not being present in
stable/train. Unfortunatley the backport of this fix is held up by
multiple other failures in other projects caused by the new pip
dependency resolver behaviour. To fix these we first need to land
I8f24b839bf42e2fb9803dc7df3a30ae20cf264eb in devstack on stable/train,
allowing fixes to land in these projects that should eventually fix
openstacksdk.

To allow this we need to temporarily mark the openstacksdk-functional
job as non-voting.

Change-Id: I8f24b839bf42e2fb9803dc7df3a30ae20cf264eb
(cherry picked from commit 09b5b05c)

Infra changed the default swap size from 8G to 1G, this breaks some
stable jobs. Revert to 8G for our jobs.

Change-Id: I4fb2372be7d861f09f365e96478b4663847a66a6

Replace legacy-tempest-dsvm-lvm-multibackend with its native version,
cinder-tempest-lvm-multibackend.

Remove legacy-tempest-dsvm-neutron-dvr-multinode-full, which was defined
as an experimental job in neutron and removed during the ussuri lifecycle.
See https://review.opendev.org/#/c/693630/

Change-Id: I76d1efaa3a6c1fe9825e8457438f514114b2ecad
(cherry picked from commit 0cbbefc0)
(cherry picked from commit c69caa4a)

A new setuptools release has changed the way pip installs are done,
see [0]. With this change we switch back to using the distro
method for global pip installs.

Temporarily make grenade jobs non-voting in order to allow this
patch to be backported.

[0] http://lists.openstack.org/pipermail/openstack-discuss/2020-August/016905.html

Change-Id: I5d8aa0e58e0409c54451b51de5eb70ba9a68d849
(cherry picked from commit 18b4251b)
(cherry picked from commit 87d1da30)

process-stackviz role currently defined in Tempest repo[1] and used
in tempest jobs. Issue with having it in branchless Tempest is that any
fix in that role cannot be backported to older Tempest. Also stackviz is
not Tempest things it is only used by Tempest jobs.

Stackviz can be considered as a service similar to Tempest so inatlling it in
Devstack side make sense. Also that will give us advantage to handle the
branch specific fixes or backpoting the fixes.

This can solve the current issues we are facing on rocky branch
- https://zuul.opendev.org/t/openstack/build/c1e2da80537448bfb24331fb4198a931/console#4/0/29/controller

[1] https://opendev.org/openstack/tempest/src/branch/master/roles/process-stackviz

Change-Id: I0ba1fd05c6391cd8bb978119ecfbb891def2d497

Add packages for openSUSE distribution for installation of uwsgi.

For train add in addition:
Add /usr/local/bin to PATH, this is not set on openSUSE and needed
to fix the setup after the pip/virtualenv changes.

Co-Authored-By: Jens Harbott <j.harbott@x-ion.de>
Change-Id: I2ce8959460a79b6472bd9dd59edd7d94eccbacf5
(cherry-picked from commit 10c3ffd2)

Fedora 29 is EOL and we no longer have nodes available for it.

Change-Id: If19b7b9858fef7422b2046cbc6c0fcd97832ebcd

Xenial jobs were failing because running virtualenv with system pip was
failing. [0] installs a working pip on xenial that can accomodate mirrors.

[0] https://review.opendev.org/736067

Change-Id: I6c75b1ccc33122f4f82427d743e1368be9bd2069