查找是否有遗漏提交
从一个分支找到所有的 commit 和 ticket 号，然后去另外一个分支去查找这些提交是否也在这个分支里。
Multibranch Pipeline
Problems
Like database product, it runs on multi-platform, but for software enginner they may only works on one platform, how they could identify their code works on all platform? manually build the various platforms? NO!
Solution
Most people would know we can use Jenkins pipeline, they may create multi Jenkins job for different stuation.
How to do it in an elegant way, I would want to share how to use multibranch pipeline to achieve.
- When create a pull request, auto parallel start simple build.
- Reviewers can decide whether to merge base on build results.
- After code merged, auto start full build.
A Code Coverage Tool - Squish Coco use examples
Preparation
- You need to ask for a free trial license and install
- You will receive a mail with username/password to login for downloading
- I test it on the Windows platform, so I download the Windows installer, then install Squish Coco and Add-in
- Installed Visual Studio 2010 or higher, I used VS2017 Professional
Code Coverage tools of C/C++
Code Coverage is a measurement of how many lines, statements, or blocks of your code are tested using your suite of automated tests. It’s an essential metric to understand the quality of your QA efforts.
Code coverage shows you how much of your application is not covered by automated tests and is therefore vulnerable to defects. it is typically measured in percentage values – the closer to 100%, the better.
When you’re trying to demonstrate test coverage to your higher-ups, code coverage tools (and other tools of the trade, of course) come in quite useful.
List of Code Coverage Tools
| Tools | Support Language | Cost | Partners |
|—|—|—|—|—|
| Squish Coco | C, C++, C#, SystemC, Tcl and QML | Not disclosed |Botom of this page Selected Clients|
| BullseyeCoverage | C, C++ |$800 for 1-year license and up | |
| Testwell| C, C++, C#, Java| Not disclosed| |
| Parasoft C/C++test |C, C++ | Not disclosed | partners |
| VECTOR Code Coverage | C, C++ | Not disclosed (free trial available)| partners |
|JaCoCo| Java | Open Source| Most famous code coverage tool in Java area |
GitSCM clone code don't display branch
最近遇到一个 regression bug，是产品完成构建之后，build commit number 不对，显示的 HEAD 而不是常见的 97b34931ac HASH number,这是什么原因呢？
我检查了 build 脚本没有发现问题，branch 的输出是正确的，那我怀疑是引入 Jenkins 的原因，果然登录到远程的 agent 上去查看分支名称如下：
C:\workspace\blog>git branch |
果然问题出在了 Jenkins 上。这个问题有简单办法解决，就是直接使用git命令来clone代码，而不使用Git插件
git clone --depth 1 -b u2opensrc https://username:"passwowrd"@git.github.com/scm/blog.git blog |
这种方式固然简单，不会出错，但它是明码显示，我岂能容忍这种不堪的处理方式吗？肯定还是要在 Git 插件上找到解决办法的。
随后google一下，果然有遇到和我一样问题的人，问题链接 这里。
他说他做了很多调查，还跟专业的 Jenkins 人士联系，试了很多次，最后找到这个办法
checkout([$class: 'GitSCM', branches: [[name: '*/feature/*']], doGenerateSubmoduleConfigurations: false, extensions: [[$class: 'LocalBranch', localBranch: "**"]], submoduleCfg: [], userRemoteConfigs: [[credentialsId: '99f978af-XXXX-XXXX-8147-2cf8f69ef864', url: 'http://TFS_SERVER:8080/tfs/DefaultCollection/Product/_git/Project']]]) |
主要是在 extensions:[] 中加入这句 [$class: ‘LocalBranch’, localBranch: “**”]
这是 Jenkins 的 Bug 吗？带着这个疑问随后通过 Pipeline Syntax，找到 checkout: Check out from version control，在 Additional Behaviours 里有 Check out to specific local branch 这个配置项
If given, checkout the revision to build as HEAD on this branch.
If selected, and its value is an empty string or “**”, then the branch name is computed from the remote branch without the origin. In that case, a remote branch origin/master will be checked out to a local branch named master, and a remote branch origin/develop/new-feature will be checked out to a local branch named develop/newfeature.
看介绍原来 Jenkins 自带这个设置，只是它不是默认选项，所以才遇到刚才那个问题。随后选择这个设置，然后填入”**”，然后生成 Pipeline 脚本，就跟上面的脚本一样了。
通过参数化上传文件到 FTP 服务器
实现 CI/CD 过程中，常常需要将构建好的 build 上传到一个公共的服务器，供测试、开发来获取最新的 build。如何上传 build 成果物到 FTP server，又不想把 FTP server登录的用户名和密码存在脚本里，想做这样的参数化如何实现呢？
upload_to_ftp.bat [hostname] [username] [password] [local_path] [remote_pat] |
Automatically commit code by Jenkins
When we need to release a product, we should change copyright, build version, release month, release note…
How to modify multiple files automatically?
I used a Jenkins pipeline project, the project is parameterized(string parameter) and regular expressions to implement.
Here is the string parameter for copyright:
Name: copyright
Default Value: 1995—2019
Description: Copyright format:1995—2019
stage('change copyrigh') {
steps {
sh label: '', script: 'sed -i -E "s/(1995—[0-9]{4})/${copyright}/" 1033/AutoRun.ini'
}
}
Here is the string parameter for build version:
Name: build_version
Default Value: 1.2.2.1002
Description: build version format: 1.2.2.1002
stage('change build version') {
steps {
sh label: '', script: 'sed -i -E "s/([0-9].[0-9].[0-9].[0-9]{4})/${build_version}/" 1033/AutoRun.ini'
}
}
Here is the string parameter for build version:
Name: release_month
Default Value: May 2019
Description: release month format: May 2019
stage('change release month') {
steps {
sh label: '', script: '''
sed -i -E "s/([a-z]* 20[0-9]{2})/${release_month}/" 1033/AutoRun.ini
sed -i -E "s/([a-z]* 20[0-9]{2})/${release_month}/" 1033/MainMenu.ini
'''
}
}
push change to Git
stage('git push to Git') {
steps {
sshagent(['8dd766ba-ac0f-4302-afa8-bee59c726dee']) {
sh("git add 1033/AutoRun.ini")
sh("git add 1033/MainMenu.ini")
sh("git commit -m 'Bld # ${build_version}'")
sh("git push -u origin master")
}
}
}Whole Jenkins Pipeline looks like:
pipeline {
agent {
label 'master'
}
stages{
stage('git clone') {
steps{
git branch: 'master',
credentialsId: '8dd766ba-ac0f-4302-afa8-bee59c726dee',
url: 'git@github.com:shenxianpeng/blog.git'
}
}
stage('change copyrigh') {
steps {
sh label: '', script: 'sed -i -E "s/(1995—[0-9]{4})/${copyright}/" 1033/AutoRun.ini'
}
}
stage('change release month') {
steps {
sh label: '', script: '''
sed -i -E "s/([a-z]* 20[0-9]{2})/${release_month}/" 1033/AutoRun.ini
sed -i -E "s/([a-z]* 20[0-9]{2})/${release_month}/" 1033/MainMenu.ini
'''
}
}
stage('change build version') {
steps {
sh label: '', script: 'sed -i -E "s/([0-9].[0-9].[0-9].[0-9]{4})/${build_version}/" 1033/AutoRun.ini'
}
}
stage('git push to Git') {
steps {
sshagent(['8dd766ba-ac0f-4302-afa8-bee59c726dee']) {
sh("git add 1033/AutoRun.ini")
sh("git add 1033/MainMenu.ini")
sh("git commit -m 'Bld # ${build_version}'")
sh("git push -u origin master")
}
}
}
}
}
Jenkins Linux agent configuration
Prepare Java runtime
Check if had installed java
$ java -version |
if not Here is an article telling you how to install it
Create Node
1. Jenkins home page->Manage Node->New Node, such as window-build-machine
2. List Linux agent settings
| Items | Settings |
|---|---|
| Name | Linux-build-machine |
| Description | used for Linux build |
| of executors | 1 |
| Remote root directory | /home/agent |
| Labels | Linux, build |
| Usage | Use this node as much as possible |
| Launch method | Launch agent agents via SSH |
| Host | 192.168.1.112 |
| Credentials | username/password |
| Host Key Verification Strategy | Manually trusted key Verification Strategy |
| Availability | Keep this agent online as much as paossible |
3. How to set credentials
| credentials | configuration |
|---|---|
| Domain | Global credentials (unrestricted) |
| Kind | Username with password |
| Scope | Global(Jenkins, nodes, items, all child items, etc) |
| Username | root |
| Password | mypassword |
| Description | Linux agent username & password |
4. Save then Connect
Remoting version: 3.29 |
Troubleshooting
| Problem | how to fix |
|---|---|
| [04/22/19 23:15:07] [SSH] WARNING: No entry currently exists in the Known Hosts file for this host. Connections will be denied until this new host and its associated key is added to the Known Hosts file. | ssh-keyscan HOSTNAME >> known_hosts |
| /var/lib/jenkins/.ssh/known_hosts [SSH] No Known Hosts file was found at /var/lib/jenkins/.ssh/known_hosts. | changing the Host key verification strategy in LAUNCH METHOD from “Known Hosts file verification strategy” to “Manually trusted key verification strategy” |
Jenkins Windows agent configuration
Prepare Java runtime
1. Download Java
2. Configure Java Windows path
JAVA_HOME=C:\Program Files\Java\jdk1.8.0_201 |
Create Node
1. Jenkins home page->Manage Node->New Node, such as window-build-machine
2. List windows agent settings
| Items | Settings |
|---|---|
| Name | window-build-machine |
| Description | used for windows build |
| of executors | 1 |
| Remote root directory | C:\agent |
| Labels | windows, build |
| Usage | Use this node as much as possible |
| Launch method | Let Jenkins control this Windows slave as a Windows service |
| Administrator user name | .\Administrator |
| Password | mypassword |
| Host | 192.168.1.111 |
| Run service as | Use Administrator account given above |
| Availability | Keep this agent online as much as paossible |
3. Save then Connect
[2019-05-11 01:32:50] [windows-slaves] Connecting to 192.168.1.111 |
Troubleshooting
The following issues I met and how I fixed them.
1. ERROR: Message not found for errorCode: 0xC00000AC
You need need to install JDK, and config JAVA environment variable.
2. How to fix add windows node as Windows service error
Ref to JENKINS-16418.
3. org.jinterop.dcom.common.JIException: Message not found for errorCode: 0x00000005
Fixed permission for the following registry keys
- HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Wow6432Node\CLSID{72C24DD5-D70A-438B-8A42-98424B88AFB8}
- HKEY_CLASSES_ROOT\CLSID{76A64158-CB41-11D1-8B02-00600806D9B6}
Steps to fix it
- Open ‘regedit’ (as Administrator), Find (Ctrl+F) the registry key: “{72C24DD5-D70A-438B-8A42-98424B88AFB8}” in HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Wow6432Node\CLSID
- Right click and select ‘Permissions’, Change owner to administrators group (Advanced…).
- Change permissions for administrators group. Grant Full Control。
- Change owner back to TrustedInstaller (user is “NT Service\TrustedInstaller” on local machine)
Repeat the above steps to fix permission for HKEY_CLASSES_ROOT\CLSID{76A64158-CB41-11D1-8B02-00600806D9B6}
Finally, Restart Remote Registry Service (Administrative Tools / Services).
4. ERROR: Unexpected error in launching an agent
This is probably a bug in Jenkins.
- Login remote machine and open Services find
jenkinsslave-C__agent - Set startup type: Automatic
- Log On: select This account, type correct account and password
- start jenkinsslave-C__agent
5. Caused by: org.jinterop.dcom.common.JIRuntimeException: Message not found for errorCode: 0x800703FA
Slave under domain account, If your slave is running under a domain account and you get an error code 0x800703FA, change a group policy:
- open the group policy editor (gpedit.msc)
- go to Computer Configuration->Administrative Templates->System-> UserProfiles, “Do not forcefully unload the user registry at user logoff”
- Change the setting from “Not Configured” to “Enabled”, which disables the new User Profile Service feature (‘DisableForceUnload’ is the value added to the registry)
6. ERROR: Message not found for errorCode: 0xC0000001 Caused by: jcifs.smb.SmbException: Failed to connect: 0.0.0.0<00>/10.xxx.xxx.xxx
Need to enable SMB1
- Search in the start menu for ‘Turn Windows features on or off’ and open it.
- Find ‘SMB1.0/CIFS File Sharing Support’ in the list of optional features that appears, and select the checkbox next to it.
- Click OK and Windows will add the selected feature.
You’ll be asked to restart your computer as part of this process.
7. .NET Framework 2.0 or later is required on this computer to run a Jenkins agent as a Windows service
Need to upgrade your .NET Framework. Here is a link for update .NET Framework.
6. more connect jenkins agent problem on windows
Please refer to this link https://github.com/jenkinsci/windows-slaves-plugin/blob/master/docs/troubleshooting.adoc
从测试到开发的五个月
每当工作闲暇，我都会时常想起好久没有更新微信公众号的文章了，总想等工作不忙的时候赶紧跟大家分享我从测试转开发这段时间的经历和感受，但工作总是有忙不完的忙，一刻都停不下来。
终于等到这一周有两天工作不是那么忙碌了，才决定将前几天写到一半的文章更新完。这是我这几个月下来感受最轻松的两天，暂时没有bug需要去调查和测试，不用去看十几年的C代码，终于有大块时间去写我负责的Python Client端代码了。这种写着代码，听着歌曲去重构，Debug，修改Unit Test Suite感觉真是幸福。
幸福的时光总是短暂的，今天就又来了两个Bug需要去调查 ε=(´ο｀*)))唉…
又把我打回原形，调查大半天之后发现原来是QA测的不对，可以松口气晚上可以不用工作更新下微信公众号了。
这五个月来，几乎每天都是白天八小时，晚上继续背着电脑回家准备继续工作，周日偶尔去公司，经常在家学习。因为角色的转变，新的项目，需要学习的地方很多。从业务到技术，再加上产品发布在即，作为一名开发新人也肩负起Bug Fix的任务，十年前的代码，全英文的文档，复杂的系统，如果不全力一搏，真担心自己转型失败，那就太打脸了。
一天的工作忙碌和压力，使得我晚上总是吃的停不下来，吃饭是我一天当中最轻松的时刻。去年我跟别人打赌减肥赢奖金，我毫无怨念的拿到了第一的奖金，可是今年再和别人打赌减肥，至今我都还没开始，马上年底了，输掉奖金是毫无悬念的。总结下来，大概是因为今年工作太忙，工作压力大的缘故，使得我无法在八小时之余安心去继续练习吉他，做keep，年假还没来得及休，真是计划不如变化快。
虽然我还是个小开发，当角色变了，角度也会有变化。
自动化测试是本分，DevOps是阶梯
这几年下来相信你也会真切感受到，如果一名测试人员不懂自动化测试，不会写自动化测试脚本，不但难有升职或是跳槽的机会，很有可能会被企业所淘汰。
个人觉得DevOps是未来一段时间很多企业要走的路，一般的二线城市能把DevOps讲明白并且实施的人太少了，所以尽早掌握实施DevOps的人，就有机会成为DevOps教练或是测试架构师这样的角色。
没有做好抗压的准备，不要去做开发
这几个月来遇到压力非常多，从刚开始的学习C语言，到C语言考核；从学习全英文的业务文档，到业务文档的分享（也是一种考核）；从调研C代码的代码覆盖率、Git分享，到调查并解决Bug；从每天的站立会汇报到每周与国外同事的例会。终于等到九月份，Title从Quality Assurance Engineer变成了Software Engineer，这其中的压力、痛苦和短暂的喜悦只有走过的人才知道。
与年龄想匹配的能力
这点非常重要，如果现在问你，你与刚毕业两三年的同行年轻人有哪些优势？如果你不能肯定和清楚的说出自己优势的话，那就要好好反思一下了。
如果从开发角度来说，我现在就是与年龄不相匹配的能力，因此测试相关的技能以及DevOps相关知识依旧是我要好好掌握的功课。
学好英语
对于国内公司来说，工作上不会用到英语，但我想说如果想在测试和开发领域有更长远发展，英文非常重要。一般最流行开源的自动化测试框架、技术、DevOps相关的工具以及搜索最有效的解决问题的方案一般都是英文。如果你的英语不好，坚持一年半载去硬啃一手英文资料，形成习惯，受益终生。