linuxea:gitlab-ci之docker镜像质量品质报告 在此前的两篇关于gitlab-ci的镜像的安全和质量的问题上做了一些简单的描述，现在就着此前的，我们在使用另外一个开源工具dive，使用dive用来对 镜像每个图层做分析，分析效率和镜像是否有浪费的空间，最后打印一个测试品质的报告。此前在另外一篇文章如何从docker镜像恢复Dockerfile提到如何简单使用dive查看dockerfile.阅读本章节，你将了解dive的基本使用和在gitlab-ci中的集成方式。我们先来看gitlab上作者给出的基本功能如下描述显示按层细分的Docker镜像内容当你在左侧选择一个图层时，将显示该图层的内容以及右侧的所有先前图层。此外，还可以使用箭头键完全浏览文件树（可以查看一些 dockerfile指令）。<当你使用dive image的时候你会进入一个交互式的接口>指出每层中发生了哪些变化已更改，已修改，添加或删除的文件在文件树中指示。可以调整此值以显示特定图层的更改，或直到此图层的聚合更改。估算“镜像效率”左下方窗格显示基本图层信息和一个实验指标，用于猜测镜像所包含的空间浪费。这可能是跨层重复文件，跨层移动文件或不完全删除文件。提供了百分比“得分”和总浪费的文件空间。快速构建/分析周期你可以构建Docker镜像并使用一个命令立即进行分析： dive build -t some-tag .你只需要docker build使用相同的dive build 命令替换你的命令。CI集成 分析和成像，并根据镜像效率和浪费的空间获得通过/失败结果。CI=true在调用任何有效的dive命令时，只需在环境中进行设置。安装centoscurl -OL https://github.com/wagoodman/dive/releases/download/v0.7.0/dive_0.7.0_linux_amd64.rpm rpm -i dive_0.7.0_linux_amd64.rpmdockerdocker run --rm -i -v /var/run/docker.sock:/var/run/docker.sock \ -e CI=true \ wagoodman/dive:v0.7 registry.linuxea.com/dev/linuxeabbs:latest使用我们主要围绕CI展开如果要查看Dockerfile或者其他层的详情，可以使用dive IMAGE,参考如何从docker镜像恢复Dockerfile[root@linuxea.com ~]# CI=true dive registry.linuxea.com/dev/linuxeabbs:latest Fetching image... (this can take a while with large images) Parsing image... Analyzing image... efficiency: 99.6914 % wastedBytes: 1858891 bytes (1.9 MB) userWastedPercent: 0.5040 % Run CI Validations... Using default CI config PASS: highestUserWastedPercent SKIP: highestWastedBytes: rule disabled PASS: lowestEfficiency Result:PASS [Total:3] [Passed:2] [Failed:0] [Warn:0] [Skipped:1]如何理解这些内容背后的含义是什么？参考github作者的回复，总体就如下3条规则efficiency：这基本上是（总镜像大小）/（总和（所有层中的字节数））。我们的想法是，如果你不删除任何文件或添加任何文件两次，那么你的“效率”为100％。如果你开始复制/删除文件，则会根据效率计算（按浪费文件的大小加权）。wastedBytes：发现在多个层上重复的原始字节数，或者发现在更高层中删除的原始字节数（因此，如果最终未使用，则可能不应该在任何层中。userWastedPercent：这基本上是效率的倒数，除了不考虑基础镜像的任何修改。更具体地说，这是wastedBytes / sum（所有层中的字节，基本镜像层除外）。集成到管道4/4 Docker_Dive: <<: *bash_init script: - Docker_Dive artifacts: name: "$CI_JOB_STAGE-$CI_COMMIT_REF_NAME" paths: [Docker_Dive.log] 函数部分 function Docker_Dive() { export PROJECT_NAME=$(echo "$CI_PROJECT_PATH_SLUG" |awk -F- '{print $2}') export IMAGE_TAG_LATEST="$REPOSITORY_URL"/"$PROJECT_NAME":latest docker run --rm -i -v /var/run/docker.sock:/var/run/docker.sock -e CI=true \ wagoodman/dive:v0.7 "$IMAGE_TAG_LATEST" |tee Docker_Dive.log } 部分截图如下：延伸阅读linuxea:如何使用gitlab-ci/cd来构建docker镜像和发布如何从docker镜像恢复Dockerfilelinuxea:gitlab-ci/cd docker容器漏洞扫描clair-scannerlinuxea:gitlab-ci/cd runner配置和安装(一)linuxea:gitlab-ci的定时任务linuxea:docker仓库harbor-https的配置和使用linuxea:白话容器之Registry与Harbor的构建和使用 (26)linuxea:Docker多阶段构建与- target和--cache-from阅读更多devopsgitlabgitlab-ci/cdjenkins

linuxea:gitlab-ci之docker-bench-security 在前面的两篇中，记录了gitlab-ci构建构建docker镜像和安全漏洞扫描，但在github上还有一个不错的项目--> Docker Bench for SecurityDocker Bench for Security是一个脚本，用于检查有关在生产中部署Docker容器的许多常见最佳实践。附：DOCKER安全性和最佳实践这其中， 检查的东西很多样化，包括：资源限制，明文密码，swarm以及其他的一些辅助性质或者参考价值的信息。阅读本章，你将了解Docker Bench for Security在gitlab-ci中的集成实践，这将有助于推动gitlab-cI自动化。运行Docker Bench有提供现成的Docker容器镜像，我们直接拿来使用。请注意，此容器正以大量特权运行- 共享主机的文件系统，pid和网络命名空间，因为基准测试的部分应用于正在运行的主机。不要忘记根据你的操作系统调整共享卷，例如它可能不使用systemddocker run -it --net host --pid host --userns host --cap-add audit_control \ -e DOCKER_CONTENT_TRUST=$DOCKER_CONTENT_TRUST \ -v /var/lib:/var/lib \ -v /var/run/docker.sock:/var/run/docker.sock \ -v /usr/lib/systemd:/usr/lib/systemd \ -v /etc:/etc --label docker_bench_security \ docker/docker-bench-security默认情况下，会检查所有的容器项目和镜像，我们要通过选项来规避一些我们想要的结果： -b optional Do not print colors -h optional Print this help message -l FILE optional Log output in FILE -c CHECK optional Comma delimited list of specific check(s) -e CHECK optional Comma delimited list of specific check(s) to exclude -i INCLUDE optional Comma delimited list of patterns within a container name to check -x EXCLUDE optional Comma delimited list of patterns within a container name to exclude from check -t TARGET optional Comma delimited list of images name to checktype一共会检查如下几项，我们通过-c参数：-c docker_daemon_configuration 来检查我们想要一个函数的结果。如果你的粒度更小 ，可以使用:-c check_2找到。[INFO] 1 - Host Configuration [INFO] 2 - Docker daemon configuration [INFO] 3 - Docker daemon configuration files [INFO] 4 - Container Images and Build File [INFO] 5 - Container Runtime [INFO] 6 - Docker Security Operations [INFO] 7 - Docker Swarm Configuration示例：仅仅检查-c docker_daemon_configuration，并且镜像只是registry.linuxea.com/dev/linuxeabbs:latest[root@linuxea.com ~]# docker run -it --net host \ --pid host \ --userns host \ --cap-add audit_control \ -e DOCKER_CONTENT_TRUST=$DOCKER_CONTENT_TRUST \ -v /var/lib:/var/lib \ -v /var/run/docker.sock:/var/run/docker.sock \ -v /usr/lib/systemd:/usr/lib/systemd \ -v /etc:/etc \ --label docker_bench_security \ docker/docker-bench-security -t registry.linuxea.com/dev/linuxeabbs:latest -c docker_daemon_configuration在或者检查某一个小项添加到管道参数：-t 指定镜像-c 指定CIS，只检查指定的项目-l 结果输出到文件(似乎不好用，用tee代替)添加新的项3/3 DockerBench_Security: <<: *bash_init script: - docker_bench_security artifacts: name: "$CI_JOB_STAGE-$CI_COMMIT_REF_NAME" paths: [docker_bench_security.log]函数部分 function docker_bench_security() { export PROJECT_NAME=$(echo "$CI_PROJECT_PATH_SLUG" |awk -F- '{print $2}') export IMAGE_TAG_LATEST="$REPOSITORY_URL"/"$PROJECT_NAME":latest docker run -i --rm --net host \ --pid host \ --userns host \ --cap-add audit_control \ -e DOCKER_CONTENT_TRUST=$DOCKER_CONTENT_TRUST \ -v /var/lib:/var/lib \ -v /var/run/docker.sock:/var/run/docker.sock \ -v /usr/lib/systemd:/usr/lib/systemd \ -v /etc:/etc \ --label docker_bench_security \ docker/docker-bench-security -t "$IMAGE_TAG_LATEST" -c container_images |tee docker_bench_security.log }运行部分结果如下：延伸阅读linuxea:如何使用gitlab-ci/cd来构建docker镜像和发布linuxea:gitlab-ci/cd docker容器漏洞扫描clair-scannerlinuxea:gitlab-ci/cd runner配置和安装(一)linuxea:gitlab-ci的定时任务linuxea:docker仓库harbor-https的配置和使用linuxea:白话容器之Registry与Harbor的构建和使用 (26)linuxea:Docker多阶段构建与- target和--cache-from阅读更多devopsgitlabgitlab-ci/cdjenkins

linuxea:gitlab-ci/cd docker容器漏洞扫描clair-scanner 一个镜像被build完成后，并不一定就是符合预期的，这往往取决于编写Dockerfile的用户的水平。尽管我们可以安排很多规则，使用很多制度去规避这些风险，但是总有一些事情发生超乎我的预料之外。为此，我们需要一些检测 镜像是否符合预期的手段。Docker镜像包含一个应用程序及其所有依赖项。由于它还包含操作系统的众多二进制文件和库，因此确保其根文件系统中不存在任何漏洞或至少不存在任何关键或主要漏洞非常重要。扫描CI / CD管道中的镜像可以降低这种安全风险。在前一篇文章中，我简单的介绍了如何使用gitlab在一个非常简单的go项目上设置CI / CD管道。管道中定义了了两个阶段：build 镜像部署事实上 ，我们可以在build镜像之前，做静态的代码分析在此过程中构建的docker镜像基于我此前 制作的镜像，但是这是一个无验证其内部的，我们需要做一些改善。阅读本章节，你将了解在CI中对镜像做一些简单的漏洞检测，在此后的文章中还继续会有此类的应用出现在CI中。添加镜像扫描到管道有几种镜像扫描解决方案，商业和开源。在本文中，我们将使用Clair和clair-scanner，2个开源工具。GitLab的以下文档提供了添加专用于镜像扫描的附加阶段的所有说明，https：//docs.gitlab.com/ce/ci/examples/container_scanning.html，它基本上运行Clair服务器，提供现有的CVE，然后二进制文件->clair-scanner检查镜像构建的每一层。上述链接中的管道语法是基于Docker的，经过尝试，发现并不适用于当下我的shell环境，于是我做了简单的修改。那现在的这个函数风格就成了这样： function image_check() { export PROJECT_NAME=$(echo "$CI_PROJECT_PATH_SLUG" |awk -F- '{print $2}') export CLAIRR=$(docker inspect --format={{.NetworkSettings.IPAddress}} clair) export DINDIRR=$(ip addr show docker0|awk 'NR==3{split($2,a,"/"); print a[1]}') export IMAGE_TAG_LATEST="$REPOSITORY_URL"/"$PROJECT_NAME":latest if [ `docker ps -a|awk '{print $NF}'|egrep "db"|wc -l` == 1 ];then echo "db is exist"; else docker run -d --name db arminc/clair-db:latest; fi if [ `docker ps -a|awk '{print $NF}'|egrep "clair"|wc -l` == 1 ];then echo "clair is exist"; else docker run -p 6060:6060 --link db:postgres -d --name clair --restart on-failure arminc/clair-local-scan:v2.0.7; fi echo -e "\033[44;37m`date +%F/%T` CVE scanning Start running now ....\033[0m" docker run --name docker-dind -i --rm -v /var/run/docker.sock:/var/run/docker.sock docker:stable-dind apk add wget ca-certificates \ && wget -qO ./clair-scanner https://github.com/arminc/clair-scanner/releases/download/v8/clair-scanner_linux_amd64 \ && chmod +x ./clair-scanner \ && echo -e "clairIP:\033[31m\033[01m[ $CLAIRR ]\033[0m\ndockerIP: \033[31m\033[01m\033[05m[ $DINDIRR ]\033[0m\nImages Name: \033[31m\033[01m\033[05m[ $IMAGE_TAG_LATEST ]\033[0m \033[0m" \ && touch clair-whitelist.yml \ && while( ! wget -q -O /dev/null http://"$CLAIRR":6060/v1/namespaces ) ; do sleep 1 ; done \ && retries=0 \ && echo -e "\033[44;37m`date +%F/%T` Waiting for clair daemon to start\033[0m" \ && while( ! wget -T 10 -q -O /dev/null http://"$CLAIRR":6060/v1/namespaces ) ; do sleep 1 ; echo -n "." ; if [ $retries -eq 10 ] ; then echo " Timeout, aborting." ; exit 1 ; fi ; retries=$(($retries+1)) ; done \ && ./clair-scanner -c http://"$CLAIRR":6060 --ip "$DINDIRR" -r gl-container-scanning-report.html -l clair.log -w clair-whitelist.yml "$IMAGE_TAG_LATEST" || true }还是使用了docker-in-docker，不过，我从shell启动，而后传递参数，而不是直接在gitlab-ci中定义docker images。注意：我这里有使用的ansible，ansible在宿主机，docker中需要安装也可以，如果这样，就意味着需要重新build docker:dind镜像。我只能寄托于gitlab-runner的executors(执行人)支持多个，当然，这是不存在的。很明显，这也不合时宜。并且，我更喜欢这样的shell风格，仅此而已。添加了这个新阶段，让我们看看它是如何运行新的管道。下面的屏幕截图显示了这个新的image_sanning阶段的部分输出。可以在顶部看到镜像的每一层都被分析。可以看到，这似乎还不错，没有明显的漏洞。摘要这篇短文说明在现有的CI / CD管道中添加简单的镜像扫描阶段并不复杂。这提供了其他信息，例如镜像包含的漏洞的数量和ID。从这些信息来看，仍然很难知道如何处理它们，但是遵循好的习惯能够避免一些问题。延伸阅读linuxea:如何使用gitlab-ci/cd来构建docker镜像和发布linuxea:gitlab-ci/cd runner配置和安装(一)linuxea:gitlab-ci的定时任务linuxea:docker仓库harbor-https的配置和使用linuxea:白话容器之Registry与Harbor的构建和使用 (26)linuxea:Docker多阶段构建与- target和--cache-from阅读更多devopsgitlabgitlab-ci/cdjenkins

linuxea:如何使用gitlab-ci/cd来构建docker镜像和发布 此前发布了一些关于gitlab和jenkins的文章，主要围绕简单的发布更新，但没有设计到docker环节，此篇将会简单的介绍如何通过gitlab-ci来做一次docker的打包和发布。阅读此章节，你将了解如何使用gitlab-ci进行docker打包镜像，以及周边的依赖使用。如果你不了解gitlab和gitlab-runner的使用，请查看此前的gitlab和gitlab-runner的配置安装部分简述我们先准备一个非常简单的代码，而后通过gitlab-ci使用docker build镜像，而后push到仓库中，在使用ansible命令到node节点来拉取镜像，完成一个非常简单的发布更新。这里需要注意的是gitlab和私有仓库的验证问题，可以参考linuxea:gitlab和gitlab-runner与harbor的配置安装的配置部分，大致可能是这样：external_url 'http://git.linuxea.com' gitlab_rails['registry_enabled'] = true gitlab_rails['registry_path'] = "/var/opt/gitlab/gitlab-rails/shared/registry" gitlab_rails['registry_api_url'] = "http://registry.linuxea.com" registry['enable'] = false尽管这只是一个简单的测试，但仍然可以通过这个过程来了解一次构建，或者说使用gitlab-ci构建发生的事情和部分细节。我将会在后续的文章中继续更新关于从docker镜像出发的问题上在做几个ci的处理，比如：镜像质量。代码准备一个go文件测试package main import ( "fmt" "net/http" ) func Hello(w http.ResponseWriter,r *http.Request) { fmt.Println("helo www.linuxea.com") fmt.Fprint(w,"20190303 www.linuxea.com") } func login(w http.ResponseWriter,r *http.Request) { fmt.Println("login") fmt.Fprint(w,"login") } func main() { http.HandleFunc("/",Hello) http.HandleFunc("/user/login",login) err := http.ListenAndServe("0.0.0.0:8080",nil) if err != nil { fmt.Println("http listen failed") } } Dockerfiledockerfile的作用将go文件复制到容器内启动即可FROM marksugar/go:alpine-go1.11.5 as linuxea MAINTAINER www.linuxea.com LABEL maintainer="www.linuxea.com" RUN mkdir /data COPY linuxea.go /data/linuxea.go COPY . /data/ WORKDIR /data CMD ["go","run","linuxea.go"]尽管这里使用的是COPY了一个文件，为了区分dockerignore的效果，我COPY .到/data下。我想我们在实际使用中仍然有必要使用.dockerignore来忽略一些文件，如下.git **/*.log **/*-log-* **/*.tar.gz **/*.war jenkins_home/*.txt Dockerfile .dockerignore *.md !info.md LICENSE CHANGELOG .gitlab-ci.ymlRegistry我们需要一个仓库来存放构建好的镜像，我这里使用的是Harbor，我在harbor创建另一个用户，而后在gitlab中设置，现在先测试一下登陆[root@linuxea.com_10_10_240_145 ~/linuxeabbs]$ echo "123.linuxea.coM" | docker login --username linuxea registry.linuxea.com --password-stdin WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store Login SucceededHarbor的构建和使用参考此前的linuxea:白话容器之Registry与Harbor的构建和使用 (26)gitlab我至少要设置三个变量在gitlab的ci构建中，分别是：CI_BUILD_TOKENCI_BUILD_USERREPOSITORY_URL这三个变量分别扮演用户，密码，Registry地址，如下：这些变量在上面的registry中提到过，是在harbor中创建好的用户，以及仓库/dev也是我提前创建好的。此后我在.gitlab-ci.yml中调用gitlab-ci在整个gitlab-ci中，只是做了两个阶段，分别是build镜像，推送到仓库，而后到节点上进行配置一个anisble，而后调用ansible pull镜像。准备ansible[linuxea-bbs] 172.25.50.17 ansible_ssh_user=root ansible_ssh_port=22992 ansible_ssh_pass=dtops.cc我编写了简单的方式来实现这个构建和发布的过程，借用了docker-compose，在stages中分为两个阶段 ：build和deploy-test。这样用的好处是，在build中可以有多个阶段树，当build执行完成后，到了deploy-test会通过手动点击完成更新。如下：stages: - build - deploy-test # 变量 variables: LANG: "en_US.UTF-8" PROJECT_HOSTGROUP: "linuxea-bbs" before_script: - echo "start now!" - export LANG=en_US.UTF-8 - export LANGUAGE=en_US:en - export LC_ALL=en_US.UTF-8 .base_init: &bash_init stage: build allow_failure: true # build镜像 1/3 docker_build: <<: *bash_init script: - docker_build artifacts: name: "$CI_JOB_STAGE-$CI_COMMIT_REF_NAME" paths: [build.json] 2/3 image_check: <<: *bash_init script: - image_check deploy: stage: deploy-test environment: name: staging url: http://172.25.50.17:8080 only: - master script: #- ansible ${PROJECT_HOSTGROUP} -m shell -a "[ -d ${WWWDIR} ]||mkdir ${WWWDIR} -p" #- ansible ${PROJECT_HOSTGROUP} -m synchronize -a 'src=./ dest=${WWWDIR}/ rsync_opts=--delete,--exclude=*.json,--exclude=*.yml,--exclude=*.git' #- ansible ${ANSIBLEGROUP} -m shell -a "chown -R 400.400 /data/wwwroot/" - ansible ${PROJECT_HOSTGROUP} -m shell -a "echo "$CI_BUILD_TOKEN" | docker login --username "$CI_BUILD_USER" "$REPOSITORY_URL" --password-stdin" - ansible ${PROJECT_HOSTGROUP} -m shell -a "cd /data/ && docker-compose pull" - ansible ${PROJECT_HOSTGROUP} -m shell -a "cd /data/ && docker-compose up -d" when: manual allow_failure: false #-----------------------Auto_Devops----------------------------- .auto_devops: &auto_devops | [[ "$TRACE" ]] && set -x function docker_build() { echo "$CI_BUILD_TOKEN" | docker login --username $CI_BUILD_USER $REPOSITORY_URL --password-stdin # docker login -u $CI_BUILD_USER -p $CI_BUILD_TOKEN $REPOSITORY_URL export PROJECT_NAME=$(echo "$CI_PROJECT_PATH_SLUG" |awk -F- '{print $2}') export PROJECT_TAG=$(echo "$CI_COMMIT_SHA" |cut -c 1-5) export IMAGE_TAG="$REPOSITORY_URL"/"$PROJECT_NAME":"$PROJECT_TAG" export IMAGE_TAG_LATEST="$REPOSITORY_URL"/"$PROJECT_NAME" export CONTAINER_IMAGE="$REPOSITORY_URL"/"$PROJECT_NAME" docker pull "$CONTAINER_IMAGE":latest || true docker build -t "$IMAGE_TAG" . docker build -t "$IMAGE_TAG_LATEST" . docker push "$IMAGE_TAG" docker push "$IMAGE_TAG_LATEST" } function image_check() { echo "image_check" } before_script: #- echo "$CI_BUILD_TOKEN" | docker login --username $CI_BUILD_USER $REPOSITORY_URL --password-stdin - *auto_devops 重要说明如上所示，其中我使用版本号码的前五位数作为每次更新的镜像标签，并且每次会build两个镜像，分别是带标签的镜像和最新的镜像，这样的好处在于，每次更新只需要重新拉取即可。而回滚则指定镜像标签即可。命令摘取出来如下：CI_PROJECT_PATH_SLUG：获取项目名称PROJECT_TAG：获取版本号IMAGE_TAG：组合变量 export PROJECT_NAME=$(echo "$CI_PROJECT_PATH_SLUG" |awk -F- '{print $2}') export PROJECT_TAG=$(echo "$CI_COMMIT_SHA" |cut -c 1-5) export IMAGE_TAG="$REPOSITORY_URL"/"$PROJECT_NAME":"$PROJECT_TAG" export IMAGE_TAG_LATEST="$REPOSITORY_URL"/"$PROJECT_NAME" export CONTAINER_IMAGE="$REPOSITORY_URL"/"$PROJECT_NAME" docker pull "$CONTAINER_IMAGE":latest || true docker build -t "$IMAGE_TAG" . docker build -t "$IMAGE_TAG_LATEST" . docker push "$IMAGE_TAG" docker push "$IMAGE_TAG_LATEST"一旦代码发生变动就会执行gitlab-ci.yml的代码。build image开始部署点击开始执行ansible现在你的docker仓库中就被上传了对应的镜像，而latest一直都只有一个的。在实际使用的时候，如果出现权限不够，可如下尝试：usermod -aG docker gitlab-runner usermod -aG root gitlab-runner提示为了确保能够拉取到镜像，你需要配置daemon.json[root@etcd2 ~]# cat /etc/docker/daemon.json { "insecure-registries": ["registry.linuxea.com"] }另外，你可能还需要配置这些私有的域名解析 。如果你有疑问，你可以留言或者加入我们的QQ讨论群延伸阅读linuxea:gitlab-ci/cd runner配置和安装(一)linuxea:gitlab-ci的定时任务linuxea:docker仓库harbor-https的配置和使用linuxea:白话容器之Registry与Harbor的构建和使用 (26)linuxea:Docker多阶段构建与- target和--cache-from阅读更多devopsgitlabjenkins

linuxea:gitlab和gitlab-runner与harbor的配置安装 配置gitlab和gitlab-runner，我们来完成一个 gitlab-ci/cd的过程，在这个过程中， 使用了docker build镜像，那么有一些配置是需要修改的。于是我简单的记录了gitlab和gitlab-runner这个安装和配置的过程。阅读此章节，你将了解gitlab和gitlab-runner的配置和安装，以及harbor私有仓库在.gitlab-ci.yml后续使用前的配置。install gitlabyum install -y curl policycoreutils-python openssh-servercurl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ee/script.rpm.sh | sudo bashyum install -y gitlab-ceIt looks like GitLab has not been configured yet; skipping the upgrade script. *. *. *** *** ***** ***** .****** ******* ******** ******** ,,,,,,,,,***********,,,,,,,,, ,,,,,,,,,,,*********,,,,,,,,,,, .,,,,,,,,,,,*******,,,,,,,,,,,, ,,,,,,,,,*****,,,,,,,,,. ,,,,,,,****,,,,,, .,,,***,,,, ,*,. _______ __ __ __ / ____(_) /_/ / ____ _/ /_ / / __/ / __/ / / __ `/ __ \ / /_/ / / /_/ /___/ /_/ / /_/ / \____/_/\__/_____/\__,_/_.___/ Thank you for installing GitLab!修改配置文件，便于docker推送registryexternal_url 'http://git.linuxea.com' gitlab_rails['registry_enabled'] = true gitlab_rails['registry_path'] = "/var/opt/gitlab/gitlab-rails/shared/registry" gitlab_rails['registry_api_url'] = "http://registry.linuxea.com" registry['enable'] = falsegitlab-ctl reconfigure开始修改密码配置gitlab-runneryum安装$ curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.rpm.sh | sudo bash $ yum install gitlab-runner或者容器的方式docker pull gitlab/gitlab-runner:alpine-v11.9.0docker run --name gitlab-runner \ -v /srv/gitlab-runner/config:/etc/gitlab-runner \ -d gitlab/gitlab-runner:alpine-v11.9.0 或者下载二进制包$ wget -O /usr/local/bin/gitlab-runner http://meftp.ds.com/gitlab-cicd/gitlab-runner-linux-amd64 $ chmod +x /usr/local/bin/gitlab-runner配置gitlab-runner使用shell作为执行在.docker-ci.yml命令的终端工具这里的信息在项目中的settings->ci/cd->Runners中查看gitlab-runner register \ --non-interactive \ --url "http://git.linuxea.com/" \ --registration-token "ES6LybttoJUmE_MbWMuZ" \ --executor "shell" \ --description "172.25.50.150" \ --tag-list "shell" \ --run-untagged \ --locked="false"如果是容器，就需要这样docker exec -it gitlab-runner gitlab-runner register Runtime platform arch=amd64 os=linux pid=28 revision=692ae235 version=11.9.0 Running in system-mode. Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/): http://git.linuxea.com/ Please enter the gitlab-ci token for this runner: mWKTPccYDUNzcLW5Byem Please enter the gitlab-ci description for this runner: [051478b648e8]: 172.25.250.247 Please enter the gitlab-ci tags for this runner (comma separated): docker_172.25.250.247_runner Registering runner... succeeded runner=mWKTPccY Please enter the executor: ssh, virtualbox, docker-ssh+machine, kubernetes, docker, docker-ssh, parallels, shell, docker+machine: docker Please enter the default Docker image (e.g. ruby:2.1): docker:dind Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded! concurrent = 1 check_interval = 0 [session_server] session_timeout = 1800 [[runners]] name = "172.25.250.247" url = "http://git.linuxea.com/" token = "NmYzEJvPYKXQQR8-ChAu" executor = "docker" [runners.docker] tls_verify = false image = "docker:dind" privileged = false disable_entrypoint_overwrite = false oom_kill_disable = false disable_cache = false volumes = ["/cache"] shm_size = 0 [runners.cache] [runners.cache.s3] [runners.cache.gcs]这两者是不一样的，那我这里用的是shell，添加完成上线后就出现如下我们需要注意的是，我们可能需要在配置中添加一个超级权限privileged，否则可能会报权限的问题，示例如下：concurrent = 1 check_interval = 0 [session_server] session_timeout = 1800 [[runners]] name = "172.25.250.247" url = "http://git.linuxea.com/" token = "x4WupVAHeuwyyZ57nDPo" executor = "shell" privileged="true" [runners.cache] [runners.cache.s3] [runners.cache.gcs]在实际使用的时候，如果出现权限不够，可如下尝试：usermod -aG docker gitlab-runner usermod -aG root gitlab-runner延伸阅读linuxea:gitlab-ci/cd runner配置和安装(一)linuxea:gitlab-ci的定时任务linuxea:docker仓库harbor-https的配置和使用linuxea:白话容器之Registry与Harbor的构建和使用 (26)阅读更多devopsgitlabjenkins

linuxea:gitlab-ci的定时任务 在此前的基于gitlab的ci/cd中，我写了几篇文章来说明他的基本用法，我将会在后续的文章中不断来完善gitlab和jenkins的用法。阅读本章，你将了解gitlab基于ci的定时任务。安装配置gitlab-runner参考官网的安装手册进行安装即可yum install gitlab-runner而后可以常用docker或者shell的方式来选择后续命令执行的环境获取token，参考此前的安装配置文章dockergitlab-runner register \ --non-interactive \ --url "http://git.ds.com/" \ --registration-token "jQj1Fi2xd53xqg3ZeMVU" \ --executor "docker" \ --docker-image alpine:3.9 \ --description "docker-runner" \ --tag-list "docker,shell" \ --run-untagged \ --locked="false"shellgitlab-runner register \ --non-interactive \ --url "http://git.xx.com/" \ --registration-token "jQj1Fi2xd53xqg3ZeMVU" \ --executor "shell" \ --description "docker-runner" \ --tag-list "docker,shell" \ --run-untagged \ --locked="false"我这里使用的shellroot@linuxea.com ~ ☸ 2019-03-14 15:48:43 $ gitlab-runner register \ > --non-interactive \ > --url "http://git.xx.com/" \ > --registration-token "jQj1Fi2xd53xqg3ZeMVU" \ > --executor "shell" \ > --description "docker-runner" \ > --tag-list "docker,shell" \ > --run-untagged \ > --locked="false" Runtime platform arch=amd64 os=linux pid=13181 revision=4745a6f3 version=11.8.0 Running in system-mode. Registering runner... succeeded runner=jQj1Fi2x Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded! 配置文件位置root@linuxea.com ~ ☸ 2019-03-14 15:49:34 $ cat /etc/gitlab-runner/config.toml concurrent = 1 check_interval = 0 [session_server] session_timeout = 1800 [[runners]] name = "docker-runner" url = "http://git.ds.com/" token = "b5819c16ec11f45e8b6a59175453ec" executor = "shell" [runners.cache] [runners.cache.s3] [runners.cache.gcs]一般情况下，配置完成我们需要重启一下，尽管提示说会生效systemctl restart gitlab-runner.service 如果不想重启，可以通过list查看，并使用status查看是否runingroot@linuxea.com ~ ☸ 2019-03-14 15:50:34 $ gitlab-runner list Runtime platform arch=amd64 os=linux pid=14411 revision=4745a6f3 version=11.8.0 Listing configured runners ConfigFile=/etc/gitlab-runner/config.toml docker-runner Executor=shell Token=b5819c16ec11f45e8b6a59175453ec URL=http://git.ds.com/root@linuxea.com ~ ☸ 2019-03-14 15:51:30 $ gitlab-runner status docker-runner Runtime platform arch=amd64 os=linux pid=14523 revision=4745a6f3 version=11.8.0 gitlab-runner: Service is running!如果没有，也可以使用gitlab-runner start docker-runner来进行启动测试计划任务我们创建一个项目用作测试，并且写一个简单的命令，命名为.gitlab-ci.ymlstages: - test 1/1 test: stage: test script: - a=$(curl -s curlip.me|awk 'NR==5{print $2" "$3}') - echo $a如下图然后我们将使用GitLab-CI安排它每天运行。现在，我们配置"日程表"(汉化后叫日程表)。CI/CD ---> 日程表--->新建计划。我们创建一个1分钟执行一次的计划。这里的定时，取决于cron配置。那么表现的形式大致如下不过，这样你会发现，他只会运行一次，而后将遵循1小时运行一次的规律,如下图这些信息在gitlab社区有过讨论，得出的结果，也就是如果一个job设置了计划，那么最小的粒度是1小时延伸阅读安装runner配置runner查看更多如果你要查看更多关于jenkins和gitlab-ci/cd的文章，请关注一下分类。gitlab-ci/cdjenkins

linuxea:gitlab-ci/cd Gitlab配置environments 回滚(十三) gitlab rollback在gitlab整个部署里面，提供了环境的定义，环境就像CI作业的标记，描述代码的部署位置。当作业将代码版本部署到环境时，会创建部署，因此每个环境都可以有一个或多个部署。GitLab会跟踪部署情况，因此我们始终可以了解服务器上当前正在部署的内容。假如使用了Kubernetes，同样可以用来协助部署，甚至于可以从gitlab中访问部署环境中的web，当然，现在并没有k8s，简单的了解下gitlab如何回滚定义环境environmentsstages: - test - deploy test: stage: test script: echo "is me test" deploy: stage: deploy-test environment: name: staging url: https://www.linuxea.com only: - master script: echo "start deploy test"name的名称并不固定deploy: stage: deploy-test environment: name: production url: https://www.linuxea.com only: - master 当运行一次后在CI/CD的界面的下拉菜单中Environments的右侧就能看到，如下在这个界面里面的红色部分是URL，点击即可直接跳转到https://www.linuxea.com ，取决于 url: https://www.linuxea.com定义的部分回滚rollback在环境部分，可以选择点进来，在右上角的(1) view deployment就如上图中的URL一样，(2) Monitoing则关联的performance(3)Rollback则回滚到所处位置Monitoing关联performance涉及到K8S，后面在做介绍部分参考：https://docs.gitlab.com/ee/ci/environments.html#monitoring-environments

linuxea:gitlab-ci/cd Gitlab和Gitlab Prometheus监控(十二) gitlab和Prometheus监控前面一篇中，有提到过使用Hygieia，Hygieia更善于集中式的展示更多的CI套件的状态。倘若我所有的CI都是自动化(特别是使用gitlab-ci)，那我们可能只需要健康容器的运行状态了，如果是jenkins那可能有些不同，这取决你使用的方式。gitlab的套包中的Prometheus自带了gitlab套件的所有的exporter，基本上开箱即用。并且在gitlab中已经完成了部分的，在之前章节的Hygieia所能暂时的部分功能(仅仅是gitlab本身)本篇简单的叙述下，gitlab和gitlab Prometheus的监控，其中涉及到Grafana，Grafana在后面会逐渐提到的更多Hygieia参考：Hygieia dashboard简单配置(十一) 简述：如果你装的是gitlab官网的包的话，Prometheus将会捆绑在软件包中，我们只需要打开它，并且重新gitlab-ctl reconfigur即可打开来进行使用，它和Prometheus单独安装所差不多，但是还是建议另外安装，当然，如过另外安装你可能需要安装其他的几个exporter几个exporter：https://github.com/prometheus/prometheus/wiki/Default-port-allocationsPrometheus监控修改配置文件[root@linuxea-VM-Node146 ~]# cat /etc/gitlab/gitlab.rb主要修改这两项prometheus_monitoring['enable'] = true 这里改成0.0.0.0比较妥当prometheus['listen_address'] = '0.0.0.0:9090'而后gitlab-ctl reconfigur，并且restart[root@linuxea-VM-Node146 ~]# gitlab-ctl reconfigure [root@linuxea-VM-Node146 ~]# gitlab-ctl restart它的配置文件在/var/opt/gitlab/prometheus下，可以在Status界面中看到配置项[root@linuxea-VM-Node146 ~]# cat /var/opt/gitlab/prometheus/prometheus.yml 在打开之前，放行端口，而后就通过IP：PORT打开Prometheus[root@linuxea-VM-Node146 ~]# iptables -I INPUT 5 -p tcp --dport 9090 -j ACCEPT官网提供的集中查询示例：％使用的内存： (1 - ((node_memory_MemFree + node_memory_Cached) / node_memory_MemTotal)) * 100％CPU负载： 1 - rate(node_cpu{mode="idle"}[5m])传输的数据： irate(node_network_transmit_bytes[5m])收到的数据： irate(node_network_receive_bytes[5m])还提供了,postgres_exporter,redis_exporter,以及gitlab的gitlab-monitorGrafana+Prometheusso，我们安装一个grafana来展示Prometheus的信息它呈现的效果大概是这样的(Grafana安装和配置实在是太简单了，就不叙述了)我已上传json文件到github上，upload即可json地址：https://raw.githubusercontent.com/LinuxEA-Mark/jenkins_gitlab_Docker/master/gitlab_monitor_linuxea_com.jsonGitlab监控另外可以通过gitlab项目中的自带的监控来看，例如，项目中的流水线CI/CD Charts，如下：也可以查看项目的提交信息，如下：以及commits信息，如下：参考：https://docs.gitlab.com/ee/administration/monitoring/prometheus/ https://gitlab.com/gitlab-org/gitlab-monitor https://docs.gitlab.com/ee/administration/monitoring/prometheus/gitlab_metrics.html#metrics-shared-directory

linuxea:gitlab-ci/cd Hygieia dashboard简单配置(十一) Hygieia他的发音hi-gee-ya，在希腊语和罗马神话中她是健康的女神/化身在Hygieia中分为五层，UI,API,DEVOPS TOOLS，数据收集，数据存储，其中：UI层(用户界面) : 是Hygieia的前端，包含用户可以查看的所有图形用户界面（GUI）元素。用户也可以在此处配置仪表板。API层 : Hygieia API层包含Hygieia API和Audit API。Hygieia API包含与源系统数据（由服务任务收集）和Internet一起使用的所有典型REST API服务。Hygieia审计API是API端点的集合，用于审计Hygieia收集器收集的CI / CD数据。该层是本地数据层和源系统数据层的抽象。DEVOPS TOOLS层 ：该层需要CI / CD管道中的大量DevOps工具。在图中，Jira，Git，Sonar和XLDeploy作为示例列出。数据收集层：收集器层从您的DevOps工具中获取数据。反过来，这些数据会显示在您的Hygieia仪表板上。您可以选择从Hygieia收集器清单中安装适用于您的DevOps工具集的收集器。数据存储层 ：Hygieia使用MongoDB作为数据存储和检索的数据库架构概述如下图：Hygieia可以描述CI/CD管道，从本质上讲Hygieia就是一个聚合器，它从团队的CI/CD管道中使用各种devops工具提取数据，使其在仪表板中更易于理解,坦白的说，便是将交付流水全过程反馈到可视化的界面中来Hygieia的仪表板简化了近乎实时查看CICD管道的能力。仪表板使DevOps工程师和管理人员能够监控代码提交到最终生产中的部署情况。在这两点之间开始（提交）到完成（prod） - 仪表板还提供有关软件操作的整体活力和性能指标的重要信息。在其中包括：组合视图，流水线视图，产品试图，云环境视图等，阅读参考：https://capitalone.github.io/Hygieia/getting_started.html本章节是对gitlab和Hygieia，相比较jenkins，还是后者更全面简单做了安装和配置实践，需要大量的开发套件才能完成，因此与gitlab-ci关系并不大，gitlab本身已经有视图，并不是如此好用而已，本篇仅提供思路，不具备任何参考价值Hygieia与gitlab安装mvn和nodeinstall mvncd /usr/local && curl -Lk http://mirror.rise.ph/apache/maven/maven-3/3.5.4/binaries/apache-maven-3.5.4-bin.tar.gz|tar xz -C ./ && ln -s apache-maven-3.5.4 apache-maven && echo "export PATH=/usr/local/apache-maven/bin:\$PATH" >> /etc/profile && source /etc/profile && echo -e "\033[32m`mvn -version` \033[0m"install nodecd /usr/local && curl -Lk https://nodejs.org/dist/v8.11.3/node-v8.11.3-linux-x64.tar.xz|tar xJ -C ./ && ln -s node-v8.11.3-linux-x64 node && echo "export PATH=/usr/local/node/bin:\$PATH" >> /etc/profile && source /etc/profile && echo -e "\033[32m`node -v` \033[0m"Hygieia install创建一个普通用户并且切换到普通用户，克隆代码并执行mvn clean install package[root@LinuxEA-VM-Node202 ~]# useradd Hygieia [root@LinuxEA-VM-Node202 ~]# su - Hygieia [Hygieia@LinuxEA-VM-Node202 ~]$ git clone https://github.com/capitalone/Hygieia.git 正克隆到 'Hygieia'... remote: Counting objects: 34942, done. remote: Compressing objects: 100% (74/74), done. remote: Total 34942 (delta 8), reused 44 (delta 3), pack-reused 34849 接收对象中: 100% (34942/34942), 71.17 MiB | 231.00 KiB/s, done. 处理 delta 中: 100% (15509/15509), done.执行mvn clean install package，这个过程可能会很长[Hygieia@LinuxEA-VM-Node202 ~]$ cd Hygieia [Hygieia@LinuxEA-VM-Node202 ~/Hygieia]$ mvn clean install package [INFO] Scanning for projects... Downloading from central: https://repo.maven.apache.org/maven2/org/ 省略一万字 [INFO] Reactor Summary: [INFO] [INFO] com.capitalone.dashboard:Hygieia 2.0.5-SNAPSHOT .... SUCCESS [ 49.924 s] [INFO] com.capitalone.dashboard:core ...................... SUCCESS [03:03 min] [INFO] com.capitalone.dashboard:api ....................... SUCCESS [ 45.683 s] [INFO] com.capitalone.dashboard:api-audit ................. SUCCESS [ 13.169 s] [INFO] com.capitalone.dashboard:rally-collector ........... SUCCESS [ 6.604 s] [INFO] com.capitalone.dashboard:artifactory-artifact-collector SUCCESS [ 4.012 s] [INFO] com.capitalone.dashboard:bamboo-build-collector .... SUCCESS [ 3.544 s] [INFO] com.capitalone.dashboard:jenkins-build-collector ... SUCCESS [ 3.855 s] [INFO] com.capitalone.dashboard:jenkins-cucumber-test-collector SUCCESS [ 2.881 s] [INFO] com.capitalone.dashboard:jenkins-codequality ....... SUCCESS [ 7.999 s] [INFO] com.capitalone.dashboard:sonar-codequality-collector SUCCESS [ 4.993 s] [INFO] com.capitalone.dashboard:aws-cloud-collector ....... SUCCESS [ 36.819 s] [INFO] com.capitalone.dashboard:udeploy-deployment-collector SUCCESS [ 2.942 s] [INFO] com.capitalone.dashboard:xldeploy-deployment-collector SUCCESS [ 2.776 s] [INFO] com.capitalone.dashboard:jira-feature-collector .... SUCCESS [ 58.628 s] [INFO] com.capitalone.dashboard:versionone-feature-collector SUCCESS [ 11.287 s] [INFO] com.capitalone.dashboard:gitlab-feature-collector .. SUCCESS [ 3.761 s] [INFO] com.capitalone.dashboard:chat-ops-collector ........ SUCCESS [ 0.929 s] [INFO] com.capitalone.dashboard:appdynamics-performance-collector SUCCESS [ 2.725 s] [INFO] com.capitalone.dashboard:bitbucket-scm-collector ... SUCCESS [ 3.123 s] [INFO] com.capitalone.dashboard:github-scm-collector ...... SUCCESS [ 2.988 s] [INFO] com.capitalone.dashboard:github-graphql-scm-collector SUCCESS [ 4.384 s] [INFO] com.capitalone.dashboard:subversion-collector ...... SUCCESS [ 21.260 s] [INFO] com.capitalone.dashboard:gitlab-scm-collector ...... SUCCESS [ 3.083 s] [INFO] com.capitalone.dashboard:hpsm-cmdb-collector ....... SUCCESS [ 3.074 s] [INFO] com.capitalone.dashboard:nexus-iq-collector ........ SUCCESS [ 3.144 s] [INFO] com.capitalone.dashboard:score-collector ........... SUCCESS [ 4.487 s] [INFO] Hygieia Publisher Plugin ........................... SUCCESS [08:25 min] [INFO] com.capitalone.dashboard:UI ........................ SUCCESS [02:40 min] [INFO] com.capitalone.dashboard:ui-tests 2.0.5-SNAPSHOT ... SUCCESS [01:44 min] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 25:09 min [INFO] Finished at: 2018-07-11T09:59:48+08:00 [INFO] ------------------------------------------------------------------------生成认证密钥生成一个core的key[Hygieia@LinuxEA-VM-Node202 ~/Hygieia/core/target]$ java -jar core-2.0.5-SNAPSHOT.jar com.capitalone.dashboard.util.Encryption SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. Your secret key is: xIleAiYTyCy1McKXZz2dL6s4+FtiaML+ Sample encrypted string with the above key for 'thisIsMyPassword' is: 0kY/r1UAMiedT2XQCWFQwhJzAWMGiA/k [Hygieia@LinuxEA-VM-Node202 ~/Hygieia/core/target]$ gitlab tokens在gitlab上生产一个 Access Tokens点击create pesonal access token将生成的Access Token保存：NsDKbGL2P9NYXFQbtMpPmongodb授权mongodb安装配置参考：https://www.linuxea.com/1848.html创建admin账户> db.createUser({"user":"admin","pwd":"admin","roles":["root"]}) Successfully added user: { "user" : "admin", "roles" : [ "root" ] }到admin库创建用户和库的授权信息> use admin switched to db admin > db.auth("admin","admin") 1开始创建linuxeacom> db.createUser({user: "linuxeacom",pwd: "123456",roles: [{role: "readWrite", db: "linuxeacom"}]}) Successfully added user: { "user" : "linuxeacom", "roles" : [ { "role" : "readWrite", "db" : "linuxeacom" } ] } > 进入linuxeacom授权linuxeacom用户> use linuxeacom switched to db linuxeacom > db.createUser({user: "linuxeacom",pwd: "123456",roles: [{role: "readWrite", db: "linuxeacom"}]}) Successfully added user: { "user" : "linuxeacom", "roles" : [ { "role" : "readWrite", "db" : "linuxeacom" } ] } > 验证> use linuxeacom switched to db linuxeacom > db.auth("linuxeacom","123456") 1 > 配置文件api部分将key加入到api配置中[Hygieia@LinuxEA-VM-Node202 ~/Hygieia/api/target]$ cat dashboard.properties dbname=linuxeacom dbhost=10.10.240.203 dbusername=linuxeacom dbpassword=123456 dbhostport=10.10.240.203:27017 dbport=27017 dbreplicaset=false server.contextPath=/api auth.secret = hygsecret server.port=8080 key=0kY/r1UAMiedT2XQCWFQwhJzAWMGiA/kgitlab配置文件gitlab生成的Access Token填写进去，key也填写进去[Hygieia@LinuxEA-VM-Node202 ~/Hygieia/collectors/feature/gitlab/target]$ cat gitlab-application.properties # Database Name dbname=linuxeacom # Database HostName - default is localhost dbhost=10.10.240.203 # Database Port - default is 27017 dbport=27017 # MongoDB replicaset #dbreplicaset=[false if you are not using MongoDB replicaset] dbreplicaset=false dbhostport=10.10.240.203:27017,127.0.0.1:27017 # Database Username - default is blank dbusername=linuxeacom # Database Password - default is blank dbpassword=123456 # Logging File location logging.file=./logs/gitlab.log #Collector schedule (required) gitlab.cron=0 0/5 * * * * #Gitlab host (optional, defaults to 'gitlab.com') gitlab.host=10.10.240.146 #Gitlab protocol (optional, defaults to 'http') gitlab.protocol=http #Gitlab port (optional, defaults to protocol default port) gitlab.port=80 #Gitlab path (optional, if your instance of gitlab requires a path) #gitlab.path= gitlab.apiToken=NsDKbGL2P9NYXFQbtMpP gitlab.commitThresholdDays=20 gitlab.key=0kY/r1UAMiedT2XQCWFQwhJzAWMGiA/k gitlab.apiVersion=4UI切换到root用户下，安装gulp[root@LinuxEA-VM-Node202 /home/Hygieia/Hygieia/UI]# npm install -g gulp npm WARN deprecated gulp-util@3.0.8: gulp-util is deprecated - replace it, following the guidelines at https://medium.com/gulpjs/gulp-util-ca3b1f9f9ac5 npm WARN deprecated graceful-fs@3.0.11: please upgrade to graceful-fs 4 for compatibility with current and future versions of Node.js npm WARN deprecated minimatch@2.0.10: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue npm WARN deprecated minimatch@0.2.14: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue npm WARN deprecated graceful-fs@1.2.3: please upgrade to graceful-fs 4 for compatibility with current and future versions of Node.js npm WARN notice [SECURITY] minimatch has the following vulnerability: 1 high. Go here for more details: https://nodesecurity.io/advisories?search=minimatch&version=0.2.14 - Run `npm i npm@latest -g` to upgrade your npm version, and then `npm audit` to get more info. npm WARN notice [SECURITY] minimatch has the following vulnerability: 1 high. Go here for more details: https://nodesecurity.io/advisories?search=minimatch&version=2.0.10 - Run `npm i npm@latest -g` to upgrade your npm version, and then `npm audit` to get more info. npm WARN notice [SECURITY] lodash has the following vulnerability: 1 low. Go here for more details: https://nodesecurity.io/advisories?search=lodash&version=1.0.2 - Run `npm i npm@latest -g` to upgrade your npm version, and then `npm audit` to get more info. /usr/local/node-v8.11.3-linux-x64/bin/gulp -> /usr/local/node-v8.11.3-linux-x64/lib/node_modules/gulp/bin/gulp.js + gulp@3.9.1 added 253 packages in 7.074s [root@LinuxEA-VM-Node202 /home/Hygieia/Hygieia/UI]# 启动在启动api之前，加密属性的配置[Hygieia@LinuxEA-VM-Node202 ~]$ java -cp ~/.m2/repository/org/jasypt/jasypt/1.9.2/jasypt-1.9.2.jar org.jasypt.intf.cli.JasyptPBEStringEncryptionCLI input="dbpassword" password=hygieiasecret algorithm=PBEWithMD5AndDES ----ENVIRONMENT----------------- Runtime: Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 25.171-b11 ----ARGUMENTS------------------- algorithm: PBEWithMD5AndDES input: dbpassword password: hygieiasecret ----OUTPUT---------------------- 6f/7fe3Mky72zulhSUu6PyujTNtHdQmy [Hygieia@LinuxEA-VM-Node202 ~]$ 在root用户下启动启动api[Hygieia@LinuxEA-VM-Node202 ~/Hygieia/api/target]$ exit 登出 [root@LinuxEA-VM-Node202 ~]# cd /home/Hygieia/Hygieia/api/target/ [root@LinuxEA-VM-Node202 /home/Hygieia/Hygieia/api/target]# [root@LinuxEA-VM-Node202 /home/Hygieia/Hygieia/api/target]# java -jar api.jar --spring.config.location=./dashboard.properties -Djasypt.encryptor.password=hygieiasecret启动scm插件gitlab[Hygieia@LinuxEA-VM-Node202 ~/Hygieia/collectors/scm/gitlab/target]$ java -jar gitlab-scm-collector-2.0.5-SNAPSHOT.jar --spring.config.name=gitlab --spring.config.location=./gitlab-application.properties启动UI[root@LinuxEA-VM-Node202 /home/Hygieia/Hygieia/UI]# gulp serve [11:07:05] Using gulpfile /home/Hygieia/Hygieia/UI/gulpfile.js [11:07:05] Starting 'build'... [11:07:05] Starting 'clean'... [11:07:05] Finished 'clean' after 57 ms [11:07:05] Starting 'assets'... [11:07:05] Starting 'themes'... [11:07:05] Starting 'fonts'... [11:07:05] Starting 'js'... [11:07:05] Starting 'views'... [11:07:05] Starting 'test-data'... [11:07:13] Finished 'themes' after 8.05 s [11:07:13] Finished 'assets' after 8.05 s [11:07:13] Finished 'test-data' after 8.1 s [11:07:13] Finished 'views' after 8.23 s [11:07:13] Finished 'js' after 8.36 s [11:07:14] Finished 'fonts' after 9.29 s [11:07:14] Starting 'html'... [11:07:14] gulp-inject 1 files into index.html. [11:07:14] gulp-inject 155 files into index.html. [11:07:14] Finished 'html' after 219 ms [11:07:14] Finished 'build' after 9.59 s [11:07:14] Starting 'serve'... [11:07:14] Finished 'serve' after 162 ms [BS] Local URL: http://localhost:3000 [BS] External URL: http://10.10.240.202:3000 [BS] Serving files from: dist/添加防火墙放行[root@LinuxEA-VM-Node202 ~]# iptables -I INPUT 6 -p tcp --dport 3000 -j ACCEPT界面配置点击右上角的Login，在弹出来的界面上点击Sign UP，注册一个即可创建一个项目 create a new dashboard，在widget management中选择添加我这里测试repo填写完成后保存选中一个查看sonar与Hygieia启动一个sonarqube快速的run一个sonarqube[root@DS-VM-Node_10_10_240_145 ~]$ docker run -d --name sonarqube --net=host -e SONARQUBE_JDBC_USERNAME=linuxeacom -e SONARQUBE_JDBC_PASSWORD=123 -e SONARQUBE_JDBC_URL=jdbc:postgresql://10.10.240.202/linuxeacom sonarqube:6.7.4通过IP和端口能够打开并且使用sonar-scanner检测一次[gitlab-runner@DS-VM-Node_10_10_240_145 ~/builds/d7f8c868/0/Hygieia_user/linuxea_app]$ docker run -v $(pwd):/root/src -v /var/run/docker.sock:/var/run/docker.sock "newtmitch/sonar-scanner" sonar-scanner -Dsonar.host.url=http://10.10.240.145:9000 -Dsonar.projectKey=linuxea_app -Dsonar.projectName=linuxea_app -Dsonar.projectBaseDir=/root/src -Dsonar.sources=./ -Dsonar.java.binaries=. 配置sonar api切换到$PATH/Hygieia/collectors/build/sonar/target/目录下创建配置文件配置文件详情请关注备注[root@DS-VM-Node202 ~]# cd /home/Hygieia/Hygieia/collectors/build/sonar/target/ [root@DS-VM-Node202 /home/Hygieia/Hygieia/collectors/build/sonar/target]# cat sonar-application.properties # Database Name dbname=linuxeacom # Database HostName - default is localhost dbhost=10.10.240.203 # Database Port - default is 27017 dbport=27017 # MongoDB replicaset dbreplicaset=false dbhostport=[10.10.240.203:27017] # Database Username - default is blank dbusername=linuxeacom # Database Password - default is blank dbpassword=123456 # Collector schedule (required) sonar.cron=0 0/5 * * * * # Sonar server(s) (required) - Can provide multiple sonar.servers[0]=http://10.10.240.145:9000 # Sonar version, match array index to the server. If not set, will default to version prior to 6.3. sonar.versions[0]=6.7 # Sonar Metrics - Required. #Sonar versions lesser than 6.3 sonar.metrics[0]=ncloc,line_coverage,violations,critical_violations,major_violations,blocker_violations,violations_density,sqale_index,test_success_density,test_failures,test_errors,tests # For Sonar version 6.3 and above sonar.metrics[0]=ncloc,violations,new_vulnerabilities,critical_violations,major_violations,blocker_violations,tests,test_success_density,test_errors,test_failures,coverage,line_coverage,sqale_index,alert_status,quality_gate_details # Sonar login credentials sonar.username=admin sonar.password=admin [root@DS-VM-Node202 /home/Hygieia/Hygieia/collectors/build/sonar/target]# 而后就在当前目录启动[root@DS-VM-Node202 /home/Hygieia/Hygieia/collectors/build/sonar/target]# java -jar sonar-codequality-collector-2.0.5-SNAPSHOT.jar --spring.config.name=sonar --spring.config.location=./sonar-application.properties 你可能会看到这样的日志回到UI添加添加一个code analysis ,在这个下拉菜单中必须存在sonar中的项目，否则则是失败的，如果失败请检查日志保存

linuxea:gitlab-ci/cd htpps配置和gitlab-runner配置(十) gitlab https配置自己签名一个证书给gitlab使用事实上我并不推荐使用https自签证书，如果还使用了runner那你肯定会明白的根证书自签开始根证书私钥文件创建[root@Linuxea-VM-Node146 /etc/gitlab/trusted-certs]# cd /etc/pki/CA/ [root@Linuxea-VM-Node146 /etc/pki/CA]#(umask 077; openssl genrsa -out /etc/pki/CA/private/cakey.pem 2048) Generating RSA private key, 2048 bit long modulus ....+++ ..................................................................+++ e is 65537 (0x10001)生成CA自己的私钥文件，cakey.pem。默认权限为600；[root@Linuxea-VM-Node146 /etc/pki/CA]# openssl req -new -x509 -key /etc/pki/CA/private/cakey.pem -out /etc/pki/CA/cacert.pem -days 36 You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Country Name (2 letter code) [XX]:cn #国家 State or Province Name (full name) []:shanghai # 身份 Locality Name (eg, city) [Default City]:shanghai # 城市 Organization Name (eg, company) [Default Company Ltd]:ca #机构i Organizational Unit Name (eg, section) []:ops # 部门名称 Common Name (eg, your name or your server's hostname) []:linuxea-gitlab.ds.com # 主机名称 Email Address []:user@163.com # 邮箱根据私钥文件生成CA自签证书。CA给自己签发一个证书；来关注下以下几个文件index.txt 创建索引文件serial 证书编号位置echo 01 > serial 自定义开始证书编号touch crlnumber 已吊销证书编号文件[root@Linuxea-VM-Node146 /etc/pki/CA]# touch index.txt serial [root@Linuxea-VM-Node146 /etc/pki/CA]# echo 01 > serial [root@Linuxea-VM-Node146 /etc/pki/CA]# touch crlnumber 以上的三个文件，是CA配置文件中定义的文件名，必须手动创建出来；以上简单自建CA配置完毕；签发gitlab证书到gitlab目录在创建一个证书，而后用上面的根证书来进行签发[root@Linuxea-VM-Node146 /etc/pki/CA]# mkdir /etc/gitlab/ssl [root@Linuxea-VM-Node146 /etc/pki/CA]# cd /etc/gitlab/ssl [root@Linuxea-VM-Node146 /etc/gitlab/ssl]# (umask 077; openssl genrsa -out linuxea-gitlab.ds.com.key 1024) Generating RSA private key, 1024 bit long modulus .......................++++++ ....................++++++ e is 65537 (0x10001)[root@Linuxea-VM-Node146 /etc/gitlab/ssl]# ll 总用量 4 -rw------- 1 root root 891 7月 9 13:10 linuxea-gitlab.ds.com.key创建申请[root@Linuxea-VM-Node146 /etc/gitlab/ssl]# openssl req -new -key ./linuxea-gitlab.ds.com.key -out ./linuxea-gitlab.ds.com.csr -days 365 You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Country Name (2 letter code) [XX]:cn State or Province Name (full name) []:shanghai Locality Name (eg, city) [Default City]:shanghai Organization Name (eg, company) [Default Company Ltd]:gitlab-linuxea Organizational Unit Name (eg, section) []:linuxea Common Name (eg, your name or your server's hostname) []:linuxea-gitlab.ds.com Email Address []:user@163.com Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []: An optional company name []:CA服务器上操作开始签发[root@Linuxea-VM-Node146 /etc/gitlab/ssl]# openssl ca -in linuxea-gitlab.ds.com.csr -out ./linuxea-gitlab.ds.com.crt -days 365 Using configuration from /etc/pki/tls/openssl.cnf Check that the request matches the signature Signature ok The organizationName field needed to be the same in the CA certificate (ca) and the request (gitlab-linuxea)解决报错，修改配置文件[root@Linuxea-VM-Node146 /etc/gitlab/ssl]# vim /etc/pki/tls/openssl.cnf 84 [ policy_match ] 85 countryName|| = optional 86 stateOrProvinceName|= optional 87 organizationName| = optional 88 organizationalUnitName| = optional 89 commonName| | = supplied 90 emailAddress| | = optional在颁发一次[root@Linuxea-VM-Node146 /etc/gitlab/ssl]# openssl ca -in linuxea-gitlab.ds.com.csr -out ./linuxea-gitlab.ds.com.crt -days 365 Using configuration from /etc/pki/tls/openssl.cnf Check that the request matches the signature Signature ok Certificate Details: Serial Number: 1 (0x1) Validity Not Before: Jul 9 05:19:07 2018 GMT Not After : Jul 9 05:19:07 2019 GMT Subject: countryName = cn stateOrProvinceName = shanghai organizationName = gitlab-linuxea organizationalUnitName = linuxea commonName = linuxea-gitlab.ds.com emailAddress = user@163.com X509v3 extensions: X509v3 Basic Constraints: CA:FALSE Netscape Comment: OpenSSL Generated Certificate X509v3 Subject Key Identifier: DE:FF:FD:A3:6E:96:9A:F8:D4:9A:1E:F2:FE:1B:99:FB:A3:A3:07:4F X509v3 Authority Key Identifier: keyid:00:8A:F1:19:5D:42:7E:CD:14:10:56:62:6C:C1:D6:00:36:6C:B0:29 Certificate is to be certified until Jul 9 05:19:07 2019 GMT (365 days) Sign the certificate? [y/n]:y 1 out of 1 certificate requests certified, commit? [y/n]y Write out database with 1 new entries Data Base Updated查看[root@Linuxea-VM-Node146 /etc/gitlab/ssl]# cat /etc/pki/CA/serial 02 [root@Linuxea-VM-Node146 /etc/gitlab/ssl]# ll /etc/pki/CA/newcerts/01.pem -rw-r--r-- 1 root root 3917 7月 9 13:19 /etc/pki/CA/newcerts/01.pem修改gitlab文件[root@Linuxea-VM-Node146 ~]# vim /etc/gitlab/gitlab.rb 13 external_url 'https://linuxea-gitlab.ds.com' 14 nginx['redirect_http_to_https']=true 15 nginx['ssl_certificate'] = "/etc/pki/CA/newcerts/01.pem" 16 nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/linuxea-gitlab.ds.com.key" [root@Linuxea-VM-Node146 ~]# gitlab-ctl reconfiguregitlab https已经配置完成gitlab runner配置gitlab一旦用来https,这可不当紧，但是runner也就需要了，不然会报错[root@Linuxea-VM-Node_10_10_240_145 ~]$ gitlab-runner verify Running in system-mode. ERROR: Verifying runner... failed runner=awSshzhi status=couldn't execute POST against https://linuxea-gitlab.ds.com/api/v4/runners/verify: Post https://linuxea-gitlab.ds.com/api/v4/runners/verify: x509: certificate signed by unknown authority ERROR: Verifying runner... failed runner=fc4a03c9 status=couldn't execute POST against https://linuxea-gitlab.ds.com/api/v4/runners/verify: Post https://linuxea-gitlab.ds.com/api/v4/runners/verify: x509: certificate signed by unknown authority--tls-ca-file来解决[root@Linuxea-VM-Node_10_10_240_145 /etc/gitlab-runner]$ gitlab-runner register --tls-ca-file ~/01.pem Running in system-mode. Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/): https://linuxea-gitlab.ds.com/ Please enter the gitlab-ci token for this runner: awSshzhiuXp1KbxmXbk3 Please enter the gitlab-ci description for this runner: [Linuxea-VM-Node_10_10_240_145.dwhd.org]: 145-runner Please enter the gitlab-ci tags for this runner (comma separated): 145-runner Whether to run untagged builds [true/false]: [false]: true Whether to lock the Runner to current project [true/false]: [true]: true Registering runner... succeeded runner=awSshzhi Please enter the executor: docker, parallels, shell, docker-ssh+machine, kubernetes, docker-ssh, ssh, virtualbox, docker+machine: shell Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded! 正确的方式这样操作完成后,可能还是不行的,将之前的文件都拿到runner机器上来创建目录后，使用tls_cert_path=/etc/gitlab-runner/certs指定即可[root@Linuxea-VM-Node_10_10_240_145 /etc/gitlab-runner/certs]$ ll 总用量 12 -rw-r--r-- 1 root root 3917 7月 26 22:14 01.pem -rw-r--r-- 1 root root 3917 7月 26 22:04 linuxea-gitlab.ds.com.crt -rw------- 1 root root 891 7月 26 22:13 linuxea-gitlab.ds.com.key [root@Linuxea-VM-Node_10_10_240_145 /etc/gitlab-runner/certs]$ gitlab-runner register tls_cert_path=/etc/gitlab-runner/certs Running in system-mode. Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/): Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/): https://linuxea-gitlab.ds.com/ Please enter the gitlab-ci token for this runner: awSshzhiuXp1KbxmXbk3 Please enter the gitlab-ci description for this runner: [Linuxea-VM-Node_10_10_240_145.dwhd.org]: test Please enter the gitlab-ci tags for this runner (comma separated): test Whether to run untagged builds [true/false]: [false]: true Whether to lock the Runner to current project [true/false]: [true]: true Registering runner... succeeded runner=awSshzhi Please enter the executor: docker+machine, kubernetes, ssh, virtualbox, parallels, shell, docker-ssh+machine, docker, docker-ssh: shell Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded! [root@Linuxea-VM-Node_10_10_240_145 /etc/gitlab-runner/certs]$ 参考:https://docs.gitlab.com/runner/configuration/advanced-configuration.html