Setup daily build for iOS project

Recently I’ve been assigned a job to setup a build machine for my team. The build machine will do daily (nightly actually) build for our iOS project. The target is easy and clear:

  • Make the build automatically with the latest code at every night.
  • Increase build number on each build.
  • Send log and app file to an internal server.
  • Send out email with log and file location to related people.

Mac Mini has been chosen as the machine to do the build, aka Build Machine:). It’s small, low price and very quiet. The OS running on it is Lion 10.7.3 with xCode 4.3 installed. Our source control system is Perforce, so the perforce client has been installed too.

To make the job more achievable, I break it down so I can figure out how many tech points I need to take care of. So this is the list / steps:

  1. Get the latest code from Perforce
  2. Find the file which contains the build number and make it editable
  3. Get latest build number and keep it somewhere
  4. Increase the build number
  5. Update the latest build number to Perforce
  6. Do a clean build
  7. Get current time and time of last automatic build
  8. Find change lists submitted to Perforce during this time period
  9. Update last automatic build time to current time
  10. Copy all change lists and app file to internal server
  11. Generate an email with all the information and send it out
  12. Make the scripts and commands run automatically every night

OK. Let’s figure out how to do them one by one. Before that, let’s setup the 2 tools we gonna use: Perforce & xCode

Perforce

Perforce could be downloaded from the official website. P4V is a visualizaion tool for you to talk to Perforce server without touch the commands. I use it to help setup the basic env. So download the P4V, install it and run it. Sync the first copy for your code in P4V will save you lots of work. Then download P4 which is a command line tool can be used in script. The downloaded P4 doesn’t has any extension. Put it somewhere you can remember, we will use the full path in the following part. Make sure make P4 executable before use it.

sudo chmod +x /full_path_to/p4

xCode

xCode 4.3 could be downloaded from App Store. The new version has a minor change compare with the previous version. You won’t be able to find the command line tool in /Developer folder. Instead, you need to go to /Applications/Xcode.app/Contents/Developer. So before we can call xcodebuild to make the build, we need to setup the path first.

sudo xcode-select -switch /Applications/Xcode.app/Contents/Developer

OK, I think we are done for this part. Now let’s setup the script.

Get the latest code from Perforce

I think this is the easiest step. One single command of P4 can make this done.

${P4HOME}/p4 -p ${P4PORT} -c ${P4CLIENT} -u ${P4USER} -P ${P4PASSWD} sync //YOUR/PERFORCE/LOCATION/...

Find the file which contains the build number and make it editable

Since the same script might be used for different projects, I make lots of local variables so they can be changed easily. To implement this step, we just need to construct the correct file location (path) and call P4 command to make it editable.

${P4HOME}/p4 -p ${P4PORT} -c ${P4CLIENT} -u ${P4USER} -P ${P4PASSWD} edit //YOUR/PERFORCE/LOCATION/${PROJECT_NAME}/${TARGET_NAME}-Info.plist

Get latest build number and keep it somewhere

In our client, we put the version number in CFBundleVersion with the format of {MainVersion}.{MajorVersion}.{MinorVersion}. Build machine only increase the MinorVersion in each build.

buildNumber=$(/usr/libexec/PlistBuddy -c "Print CFBundleVersion" ${PROJECT_NAME}/${TARGET_NAME}-Info.plist)
IFS='.'
set $buildNumber
MAJOR_VERSION="${1}.${2}"
MINOR_VERSION="${3}"
mbm=$(($MINOR_VERSION + 1))
buildNumber="${MAJOR_VERSION}.${mbm}"

Increase the build number

After we get the number from the -Info.plist file and increase it, we write it back.

/usr/libexec/PlistBuddy -c "Set :CFBundleVersion $buildNumber" ${PROJECT_NAME}/${TARGET_NAME}-Info.plist

Update the latest build number to Perforce

Now, we need to check in new build number to Perforce.

${P4HOME}/p4 -p ${P4PORT} -c ${P4CLIENT} -u ${P4USER} -P ${P4PASSWD} submit -d "Update version number to $buildNumber"

Do a clean build

If everything goes well, the build number is now larger than the previous version. Since everything has been setup in xCode IDE, the only thing we need to do is call xcodebuild command. But I’d like to clean it first.

cd $PROJECT_NAME
xcodebuild -project ${PROJECT_NAME}.xcodeproj -target ${TARGET_NAME} clean
xcodebuild -project ${PROJECT_NAME}.xcodeproj -target ${TARGET_NAME}
cd ..

You might notice that I haven’t check if the build is success or not. Yes, in the first version, I’d like to keep it simple. I will go back to add more features later.

Get current time and time of last automatic build

This is easy. I simply keep the date and time of last build in a text file and get current time from script. The time format is YYYY/MM/DD:HH:MM:SS.

lastBuildDateTime=$(cat "LastBuildDateTime.txt")
curDate=$(date +"%Y-%m-%d")
curDateTime=$(date +"%Y/%m/%d:%T")

Find change lists submitted to Perforce during this time period

With the last build time and current time, we are able to get all check-ins in this time range. With these information, QA could know what’s the different between builds. BTY, I output the changes to a text file directly b/c I found it’s not easy to keep the format returned from Perforce server. Sending them to file directly keeps the format.

buildTargetName=Build-$curDate-$mbm
${P4HOME}/p4 -p ${P4PORT} -c ${P4CLIENT} -u ${P4USER} -P ${P4PASSWD} changes -l -t -s submitted //YOUR/PERFORCE/LOCATION/...@$lastBuildDateTime,@now >> $buildTargetName.txt

Update last automatic build time to current time

# Use > to force overwrite the original file
echo $curDateTime > LastBuildDateTime.txt

Copy all change lists and app file to internal server

With the help of cp/mkdir commands, we can make this happen easily. In the Build Machine, I mount the server folder when the computer starts. So I can reference the path directly in my script.

# Copy change list file to server
mkdir YOUR/SERVER/PATH/Builds/${SERVER_SUB_FOLDER}
mkdir YOUR/SERVER/PATH/Builds/${SERVER_SUB_FOLDER}/$buildTargetName
cp $buildTargetName.txt YOUR/SERVER/PATH/Builds/${SERVER_SUB_FOLDER}/$buildTargetName/

# Copy app file to server
mkdir YOUR/SERVER/PATH/Builds/${SERVER_SUB_FOLDER}/$buildTargetName/Build
cp -r LOCAL/PATH/Release-iphoneos/* YOUR/SERVER/PATH/Builds/${SERVER_SUB_FOLDER}/$buildTargetName/Build

# Copy provision file
cp dev.mobileprovision YOUR/SERVER/PATH/Builds/${SERVER_SUB_FOLDER}/$buildTargetName/Build

Generate an email with all the information and send it out

Thanks Apple for the great email feature in Lion. We can do this quite easy.

# Send email
# Construct mail content
echo "Hi, a new build of ${PROJECT_NAME}-${TARGET_NAME} is ready." >> msg.txt
echo " " >> msg.txt
echo "The change list in this build is:" >> msg.txt
echo " " >> msg.txt
${P4HOME}/p4 -p ${P4PORT} -c ${P4CLIENT} -u ${P4USER} -P ${P4PASSWD} changes -l -t -s submitted //YOUR/PERFORCE/LOCATION/...@$lastBuildDateTime,@now >> msg.txt
echo " " >> msg.txt
echo "New build is available at YOUR/SERVER/PATH/Builds/${SERVER_SUB_FOLDER}/$buildTargetName" >> msg.txt
echo " " >> msg.txt
echo "Thanks." >> msg.txt
echo " " >> msg.txt
echo "Zunlin Zhang (sent from iOS Build Machine)" >> msg.txt

# Set up mail list and send it
subject="["$curDate"] New build for ${PROJECT_NAME}-${TARGET_NAME} is ready, build number: ${buildNumber}"
emails="qa@mycompany.com,dev@gmycompany.com"
/usr/bin/mail -s "$subject" "$emails" < "msg.txt"

Make the scripts and commands run automatically every night

This part costs me almost a whole day. Mac OS has something named Launhd which is:

a unified, open-source service management framework for starting, stopping and managing daemons, applications, processes, and scripts. Written and designed by Dave Zarzycki at Apple, it was introduced with Mac OS X Tiger and is licensed under the Apache License.

You might know crontab in Linux can do the same thing. But Apple suggest you move your tasks to Launchd. This blog helps me setup my daily task successfully. And if you have trouble to write your own plist configuration file, Lingon can help generate it easily. This is my plist file, and I put it in $HOME/Library/LaunchAgents.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>AbandonProcessGroup</key>
	<true/>
	<key>KeeyAlive</key>
	<true/>
	<key>Label</key>
	<string>com.ios.DailyBuild</string>
	<key>ProgramArguments</key>
	<array>
		<string>/bin/sh</string>
		<string>DailyBuild.sh</string>
	</array>
	<key>RunAtLoad</key>
	<true/>
	<key>StandardErrorPath</key>
	<string>error.txt</string>
	<key>StartCalendarInterval</key>
	<dict>
		<key>Hour</key>
		<integer>21</integer>
		<key>Minute</key>
		<integer>0</integer>
	</dict>
</dict>
</plist>

The key here is: <KeeyAlive> and <AbandonProcessGroup>. The first one makes the script  always running and the second one prevents sub-threads being killed after the main script is done. What does this mean? At the end of my script, I send out email with the build information. I guess sending email might not happens in the main thread. So the script run ends after “send email” operation is fired. But the email is still being sent in the background. If we kill the main thread at the time, email might not be sent out in the end. With <AbandonProcessGroup> set to true, system will wait for all sub-threads finish to kill the main script.

Notes

I believe you notice that I include all Perforce parameters in each P4 command. They should stay in a global setting like .bash_profile. But I found it didn’t work with Launchd if I use the global setting. I think I must miss something but I don’t know which one. Put them directly in the code fix this issue.

The other thing you might notice is when I get the change lists from Perforce, I use @now parameter. But the start time is generate from local time. If your Perforce server runs in a different time zone, you might have trouble. My solution is to change the local time zone to whatever my Perforce server uses.

Making a daily build machine reduces lots of work from Engineers. My scripts here are very junior and needs to be improved in the future. But it covers the basic requires my team needs now:)

发表评论

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / 更改 )

Twitter picture

You are commenting using your Twitter account. Log Out / 更改 )

Facebook photo

You are commenting using your Facebook account. Log Out / 更改 )

Google+ photo

You are commenting using your Google+ account. Log Out / 更改 )

Connecting to %s