Execution context selection

Some tests are designed to be run against specific environments, or in specific pipelines or jobs. We can specify the test execution context using the only and except metadata.

Available switches

WARNING: You cannot specify :production and { <switch>: 'value' } simultaneously. These options are mutually exclusive. If you want to specify production, you can control the tld and domain independently.

Examples

Only

Run tests in only the specified context.

Matches use:

• Regex for environments.
• String matching for pipelines.
• Regex or string matching for jobs
• Lambda or truthy/falsey value for generic condition

RSpec.describe 'Area' do
  it 'runs in any environment or pipeline' do; end
  it 'runs only in production environment', only: :production do; end

  it 'runs only in staging environment', only: { subdomain: :staging } do; end

  it 'runs in dev environment', only: { tld: '.org', domain: 'gitlab', subdomain: 'dev' } do; end

  it 'runs in prod and staging environments', only: { subdomain: /(staging.)?/, domain: 'gitlab' } {}

  it 'runs only in nightly pipeline', only: { pipeline: :nightly } do; end

  it 'runs in nightly and canary pipelines', only: { pipeline: [:nightly, :canary] } do; end

  it 'runs in specific environment matching condition', only: { condition: -> { ENV['TEST_ENV'] == 'true' } } do; end
end

Except

Run tests in their typical contexts except as specified.

Matches use:

• Regex for environments.
• String matching for pipelines.
• Regex or string matching for jobs
• Lambda or truthy/falsey value for generic condition

RSpec.describe 'Area' do
  it 'runs in any execution context except the production environment', except: :production do; end

  it 'runs in any execution context except the staging environment', except: { subdomain: :staging } do; end

  it 'runs in any execution context except the nightly pipeline', except: { pipeline: :nightly } do; end

  it 'runs in any execution context except the ee:instance job', except: { job: 'ee:instance' } do; end

  it 'runs in specific environment not matching condition', except: { condition: -> { ENV['TEST_ENV'] == 'true' } } do; end
end

Usage notes

If the test has a before or after block, you must add the only or except metadata to the outer RSpec.describe block.

To run a test tagged with only on your local GitLab instance, you can do one of the following:

• Make sure you do not have the CI_PROJECT_NAME or CI_JOB_NAME environment variables set.
• Set the appropriate variable to match the metadata. For example, if the metadata is only: { pipeline: :nightly } then set CI_PROJECT_NAME=nightly. If the metadata is only: { job: 'ee:instance' } then set CI_JOB_NAME=ee:instance.
• Temporarily remove the metadata.

To run a test tagged with except locally, you can either:

• Make sure you do not have the CI_PROJECT_NAME or CI_JOB_NAME environment variables set.
• Temporarily remove the metadata.

Quarantine a test for a specific environment

Similarly to specifying that a test should only run against a specific environment, it's also possible to quarantine a test only when it runs against a specific environment. The syntax is exactly the same, except that the only: { ... } hash is nested in the quarantine: { ... } hash. For example, quarantine: { only: { subdomain: :staging } } only quarantines the test when run against staging.

The quarantine feature can be explicitly disabled with the DISABLE_QUARANTINE environment variable. This can be useful when running tests locally.