0

Summary

My gitlab-ci.yml has 3 stage for deploy an application to okd pod Application running spring boot on tomcat:8 Sometimes, the cache.zip is not update after stage complete so that the next step can't run correctly

Steps to reproduce

My gitlab-ci run the following stage

Stage 1: run test compile ---> OK

Stage 2: package war file as output for deploy ---> Gitlab-ci log show success but the cache.zip has not war file (just sometimes cache.zip not have war file, sometimes it run correctly)

Stage 3: Deploy war file to pod ---> Because of war file not exists in cache.zip, script error -> failed

.gitlab-ci.yml

image: openshift/origin-cli

stages:
  - build
  - test
  - staging

cache:
  paths:
    - .m2/repository
    - target
    - artifact

validate:jdk8:
  stage: build
  script:
    - 'mvn test-compile'
  only:
    - master
  image: maven:3.3.9-jdk-8

verify:jdk8:
  stage: test
  script:
    - 'mvn verify'
    - 'mvn package' # =====> this command generate war file
  only:
    - master
  image: maven:3.3.9-jdk-8

staging:
  script:
    - "mkdir -p artifact"
    - "cp ./target/*.war ./artifact/" # ======> Sometimes error at this line because of previous step not add war file into cache
    - "oc start-build $APP"
    - "rm -rf ./target/* && rm -rf ./artifact/*" # Remove war & class file, only cache m2 lib
  stage: staging
  variables:
    APP: $CI_PROJECT_NAME
  environment:
    name: staging
    url: http://$CI_PROJECT_NAME-staging.$OPENSHIFT_DOMAIN
  only:
    - master

Actual behavior Sometimes cache not have war file after test stage complete (is this depends on war file size?)

Expected behavior War file update into cache after test stage for staging stage deploy

Relevant logs and/or screenshots

ScreenShot

job log

Running with gitlab-runner 13.7.0 (943fc252)
  on gitlab-runner-node1 y6awygsj
Preparing the "docker" executor
00:01
Using Docker executor with image openshift/origin-cli ...
Using locally found image version due to if-not-present pull policy
Using docker image sha256:7ebb6be01117a50344d63f77c385a13302afecd33480b97c36a518d4f5ebc25a for openshift/origin-cli with digest docker.io/openshift/origin-cli@sha256:509e052d0f2d531b666b7da9fa49c5558c76ce5d286456f0859c0a49b16d6bf2 ...
Preparing environment
00:00
Running on runner-y6awygsj-project-489-concurrent-0 via gitlab.runner.node1...
Getting source from Git repository
00:01
Fetching changes...
Reinitialized existing Git repository in /builds/my-project/.git/
Checking out b4c97428 as master...
Removing .m2/
Removing artifact/
Removing target/
Skipping Git submodules setup
Restoring cache
00:05
Checking cache for default-23...
No URL provided, cache will not be downloaded from shared cache server. Instead a local version of cache will be extracted. 
Successfully extracted cache
Executing "step_script" stage of the job script
00:01
$ mkdir -p artifact
$ cp ./target/*.war ./artifact/
cp: cannot stat './target/*.war': No such file or directory
Cleaning up file based variables
00:00
ERROR: Job failed: exit code 1

Environment description

config.toml

concurrent = 1
check_interval = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "gitlab-runner-node1"
  url = "https://gitlab.mycompany.vn/"
  token = "y6awygsj9zks18nU6PDt"
  executor = "docker"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
    [runners.cache.azure]
  [runners.docker]
    dns = ["192.168.100.1"]
    tls_verify = false
    image = "alpine:latest"
    privileged = false
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/mnt/nfs/nfsshare-gitlab/cache:/cache"]
    shm_size = 0
    pull_policy = "if-not-present"

Used GitLab Runner version

Version: 13.7.0

Git revision: 943fc252

Git branch: 13-7-stable

GO version: go1.13.8

Built: 2020-12-21T13:47:06+0000

OS/Arch: linux/amd64

Possible fixes

Re-run test stage until cache has war file

4

1 回答 1

0

让我们一步一步来。

首先，关于如何管理阶段之间的文件。

确实，如果两者都在同一环境中运行，您可以直接访问作业和阶段之间的文件，但情况并非总是如此（即使两个运行器都使用相同的 nfs 共享目录），您应该使用artifacts它。

当您artifact在作业内定义时，您指定的是在作业成功、失败或始终附加到作业的文件列表，具体取决于您的配置。

默认情况下，artifacts之前阶段的所有内容都将传递给每个作业，但在任何情况下，您都可以使用dependencies它来定义要从哪些作业中获取工件。

所以基本上你应该使用以下.gitlab-ci.yml

image: openshift/origin-cli

stages:
  - build
  - test
  - staging

cache:
  paths:
    - .m2/repository

validate:jdk8:
  stage: build
  script:
    - 'mvn test-compile'
  only:
    - master
  image: maven:3.3.9-jdk-8

verify:jdk8:
  stage: test
  script:
    - 'mvn verify' # =====> verify already includes: validate, compile, test and package
  artifacts:
    paths:
      - target/[YOUR_APP_NAME].war
  only:
    - master
  image: maven:3.3.9-jdk-8

staging:
  dependencies:
    - verify:jdk8
  script:
    - "mkdir -p artifact"
    - "cp ./target/[YOUR_APP_NAME].war ./artifact/"
    - "oc start-build $APP"
  stage: staging
  variables:
    APP: $CI_PROJECT_NAME
  environment:
    name: staging
    url: http://$CI_PROJECT_NAME-staging.$OPENSHIFT_DOMAIN
  only:
    - master

另外，请注意我删除了该mvn package说明。我建议您查看Maven 的构建生命周期基础知识。

于 2021-10-21T11:20:07.473 回答