Last week Microsoft released the schedule for sessions for Build 2020. However, other than via the portal, there’s no easy way to get a list of sessions so that you can easily work out which sessions work for your time-zone and which sessions you want to attend. In order to solve my own dilemma of … Continue reading “Add Build 2020 Schedule to Your Calendar”
Last week Microsoft released the schedule for sessions for Build 2020. However, other than via the portal, there’s no easy way to get a list of sessions so that you can easily work out which sessions work for your time-zone and which sessions you want to attend. In order to solve my own dilemma of how to add the sessions to my calendar I built a quick service that would generate the appropriate ICS files. Actually my first option was a single ICS with all sessions, hence there are two options:
Here’s how the whole thing went down – mid-last week I was wondering whether there was an easy way to add the schedule to my Outlook calendar. At this point the only schedule that had been announced was a high level list of just some of the sessions eg:
Given there’s not that many sessions, it would have been quick enough for me to add them individually to Outlook. But no, the developer in me went hunting for how to automate this. Opening Fiddler I saw that there was indeed an API that was being called at endpoints beginning with https://api.mybuild.microsoft.com. However, at this point there weren’t any calls to retrieve sessions, speakers etc, the only calls were to the /chrome and /settings endpoints which didn’t really reveal much.
Thinking that I wouldn’t be able to get much further I tried a last ditch effort of searching for other uses on the web for api.mybuild.microsoft.com. I guessed there wouldn’t be any direct hits, other than for the mybuild.microsoft.com portal, but I figured that given the way that search engines look for similar matches, something might turn up. Sure enough, the api url is not to dissimilar to the one used for prior events eg https://api.mybuild.techcommunity.microsoft.com. This lead me to a pretty sophisticated PowerShell script that Michel de Rooij had put together for scraping session content for prior events.
Knowing that Microsoft typically uses the same vendor for things to make it easier year-on-year, it’s no surprising that the api described by Michel’s PowerShell script was again being used for Build 2020.
I quickly used Json2Csharp QuickType (I can never remember this name but the service is great!) to generate the classes required to interact with the API. I can’t say that I engineered the ASP.NET Core 3.1 Web API service to be particularly efficient, mainly because I predicted that I was going to put both Azure CDN and CloudFlare in front of it. It basically pulls requests the entire list of sessions and holds them in a static variable. The only caching being that if the service is requested again whilst the static variable hasn’t been collected, it will return the previously retrieved list of sessions (do NOT follow this architecture for a production application).
I used iCal.NET to generate the ICS files – initially one monster ICS which has all the sessions but then I realised that it might be better to have individual ICS files that were downloaded as a Zip file. The ICS file(s) also contain session speakers (with links to their profile and/or their company, where available) and a link to the session for registering and attending.
Next I published the ASP.NET Core service to a new Azure App Service. As this was a personal project I opted for a free Azure App Service Plan. Of course, this will not scale well, since it’s really designed for development and testing purposes. However, given that I’m generating ICS files that aren’t likely to change frequently, I figure I’m going to protect my service so that it doesn’t get slammed.
The protection comes in two forms:
Firstly, I added an Azure CDN endpoint in front of the app service – this will cache the output of my service (level 1)
Secondly, I added Cloud Flare for two reasons. This gives me a nice friendly url (ie build2020.builttoroam.com) that I can share, without exposing either my Azure CDN endpoint or my app service url. I also enabled the proxying option which means that CloudFlare also caches the output of my service (level 2).
The upshot is that even if a massive number of people download the ICS file(s), my service shouldn’t even need to wake up. This is reflected by the analytics from my Azure App Service that shows almost zero out-bound data over the last 24 hours.
We just released v0.5.1 of the Pipeline Templates – templates for creating build pipelines for Azure DevOps. This was a minor increment but fixed an issue that emerged due to a change in the validation of pipelines by Azure DevOps. Before we get into the details of what is included in v0.5.1, the other big … Continue reading “Pipeline Templates: Stage Dependency Fix and Improved Docs”
We just released v0.5.1 of the Pipeline Templates – templates for creating build pipelines for Azure DevOps. This was a minor increment but fixed an issue that emerged due to a change in the validation of pipelines by Azure DevOps.
Before we get into the details of what is included in v0.5.1, the other big news is that Pipeline Templates has a new documentation site.
This site uses GitHub Pages and as such is generated from the markdown files in the project itself. Currently the site has a getting started guide, followed by a summary of each of the templates. Hopefully over time this can grow as we include more examples and extend the list of templates.
Pipeline Templates v0.5.1
Here’s a summary of what’s in this release:
Fix: build-xamarin-[iOS/android/windows].yml and deploy-appcenter.yml – changed depends_on parameter from stageList to string to fix validation issue introduced by Azure DevOps
Updated deploy-appcenter.yml to include additional parameters for controlling how apps are deployed via AppCenter. For example you can now specify release notes as a file, and you can flag whether a release is a mandatory update or not.
My last post was a bit of a long one as it covered a bunch of steps for setting up the bits and pieces required for signing an application for different platforms. In this post I just wanted to provide a complete example that shows a single multi-stage (6 in total) Azure Pipelines pipeline for building a Uno application for iOS, Android and Windows (UWP) and releasing them to App Center.
In my last post I showed how to create and populate Secure Files in Azure Pipelines. Any certificate or provisioning profile you need to use in your pipeline should be added to the Secure Files section of the Library in Azure Pipeline. My list of Secure Files looks like this:
Here we can see that we have the signing certificates for iOS and Windows, and the keystore for Android. Then we have two iOS provisioning profiles, one for my XF application and the other for my Uno application.
I’ve extracted most of the variables I use in my pipeline into one of two variable groups:
The Common Build Variables are those variables that can be reused across multiple projects.
The Inspector Uno Build Variables are those variables that are specific to this project. For example it includes the AppCenter ids for the iOS, Android and Windows applications. It also includes the iOS provisioning profile which is specifically tied to this application.
Here’s the entire pipeline:
resources: repositories: - repository: builttoroam_templates
ref: refs/tags/v0.5.0 endpoint: github_connection
variables: - group:'Common Build Variables' - group:'Inspector Uno Build Variables' - name: ios_enabled
value:'true' - name: windows_enabled
value:'true' - name: android_enabled
value:'true'stages:- template: azure/mobile/build-xamarin-android.yml@builttoroam_templates
parameters:# Stage name and whether it's enabled stage_name:'Build_Android' build_android_enabled: $(android_enabled)
# Version information full_version_number:'$(version_prefix).$(Build.BuildId)'# Signing information secure_file_keystore_filename:'$(android_keystore_filename)' keystore_alias:'$(android_keystore_alias)' keystore_password:'$(android_keystore_password)'# Solution to build solution_filename: $(solution_file)
# Output information artifact_folder: $(artifact_android_folder)
- template: azure/mobile/deploy-appcenter.yml@builttoroam_templates
parameters:# Stage name and dependencies stage_name:'Deploy_Android' depends_on:'Build_Android' deploy_appcenter_enabled: $(android_enabled)
# Build artifacts artifact_folder: $(artifact_android_folder)
# Signing information (for Android repack to APK) secure_file_keystore_filename:'$(android_keystore_filename)' keystore_alias:'$(android_keystore_alias)' keystore_password:'$(android_keystore_password)'# Deployment to AppCenter appcenter_service_connection: $(appcenter_service_connection)
- template: azure/mobile/build-xamarin-windows.yml@builttoroam_templates
parameters:# Stage name and whether it's enabled stage_name:'Build_Windows' build_windows_enabled: $(windows_enabled)
# Version information full_version_number:'$(version_prefix).$(Build.BuildId)'# Signing information windows_cert_securefiles_filename:'$(windows_signing_certificate_securefiles_filename)' windows_cert_password:'$(windows_signing_certificate_password)'# Solution to build solution_filename: $(solution_file)
# Output information artifact_folder: $(artifact_windows_folder)
- template: azure/mobile/deploy-appcenter.yml@builttoroam_templates
parameters:# Stage name and dependencies stage_name:'Deploy_Windows' depends_on:'Build_Windows' deploy_appcenter_enabled: $(windows_enabled)
# Build artifacts artifact_folder: $(artifact_windows_folder)
# Deployment to AppCenter appcenter_service_connection: $(appcenter_service_connection)
- template: azure/mobile/build-xamarin-ios.yml@builttoroam_templates
parameters:# Stage name and whether it's enabled stage_name:'Build_iOS' build_ios_enabled: $(ios_enabled)
# Version information full_version_number:'$(version_prefix).$(Build.BuildId)'# Solution to build solution_filename: $(solution_file)
# Signing information ios_plist_filename:'src/Apps/DotNet/Uno/InspectorUno/InspectorUno/InspectorUno.iOS/Info.plist' ios_cert_password:'$(ios_signing_certificate_password)' ios_cert_securefiles_filename:'$(ios_signing_certificate_securefiles_filename)' ios_provisioning_profile_securefiles_filename:'$(ios_provisioning_profile_securefiles_filename)'# Output information artifact_folder: $(artifact_ios_folder)
- template: azure/mobile/deploy-appcenter.yml@builttoroam_templates
parameters:# Stage name and dependencies stage_name:'Deploy_iOS' depends_on:'Build_iOS' deploy_appcenter_enabled: $(ios_enabled)
# Build artifacts artifact_folder: $(artifact_ios_folder)
# Deployment to AppCenter appcenter_service_connection: $(appcenter_service_connection)
Pipeline Templates v0.5.0
Most of the v0.5.0 release has been tidying things up, reducing the number of required parameters by making the template smarter and increasing consistency across the templates
build-xamarin-android.yml – changed build_platform to solution_target_platform parameter
build-xamarin-windows.yml – changed build_platform to solution_target_platform parameter
build-xamarin-windows.yml – changed windows_appxupload_name parameter to windows_upload_name to reflect support for msix
build-xamarin-[iOS/android/windows].yml – added depends_on parameter so that the stages can be ordered
build-xamarin-[iOS/android/windows].yml – artifact_name, artifact_folder and application_package are no longer required and have default values
build-xamarin-android.yml – supports building either aab or apk based on the application_package parameter
build-xamarin-windows.yml – windows_upload_name parameter isn’t required as it has a default value based on the bundle name
deploy-appcenter.yml – added conditions to all steps so that pipeline doesn’t break if build stage doesn’t generate any output
All template – documentation added to parameters and steps
Wow, that title’s a mouthful, and I didn’t add in there that I’ve just pushed v0.2.0 release of the Pipeline Templates repository. In this post we’re going to add stages to a YAML based Azure DevOps pipeline in order to deploy a Xamarin.Forms application to AppCenter for testing. We’ll also be using on the of the latest features of the Azure DevOps YAML based pipelines, Environments, to insert a manual approval gate into our multi-stage pipeline.
Pipeline Templates v0.2.0
Before we get into talking about releasing apps to AppCenter I just wanted to reiterate that there is a new release of the Pipeline Templates repository with the following changes:
Added a new parameter, stage_name, to iOS, Android and Windows build templates. It has a default value, which matches the value previously specified on the stage element in the template, so won’t break any existing builds. This parameter can be set by the calling pipeline so that the stage is given a known name, which can then be referenced by other stages in the dependsOn element.
Added deployappcenter.yml template that can be used to deploy iOS, Android and Windows apps to AppCenter. For Android, if the application_package parameter is an .aab file, the calling pipeline will also need to supply keystore information. AppCenter doesn’t support .aab files, so the pipeline uses bundletool to generate and sign a fat apk, which is submitted to AppCenter.
Deploy to AppCenter
With a classic pipelines in Azure DevOps, you can setup separate build and release pipelines. However, YAML pipelines don’t differentiate between build and release pipelines; instead you can split a single pipeline into multiple stages (as we demonstrated in the previous post when we used different templates to create different stages in the build process).
To deploy apps to AppCenter we could simply create a template, similar to what we did with the build templates, that includes the tasks necessary to deploy to AppCenter, and then add new stages to our pipeline for each app we want to deploy. However, this would limit our ability to take advantage of some of the deployment specific features that are available in Azure DevOps. For this reason, the AppCenter template makes use of a deployment job in order to do the steps necessary to release an app to AppCenter.
Using the AppCenter Template
The following YAML pipeline provides an example of using the new AppCenter deployment template to deploy Windows (UWP), Android and iOS applications to AppCenter. Note that the ref element for the repository resource has been updated to point to the v0.2.0 tag.
There are a couple of prerequisites that need to be setup in order for the deploy stages to work:
Each deploy stage specifies a depends_on property. This needs to correlate to the stage_name property specified on the corresponding build stage.
The artifact name, artifact folder and application_package properties need to match to the values used in the corresponding build stage.
A Service Connection needs to be established between Azure DevOps and AppCenter (in this case it was given the name ‘AppCenterInspectorCI’)
An application needs to be registered in AppCenter for each target platform. Each deploy stage needs the organisation (ie username or org_name) and applicationid (app_identifier). See the documentation on the AppCenterDistribute task for more information on how to find these values for your AppCenter apps.
Manual Approval with Environments
I mentioned earlier that using a deployment job would allow us to take advantage of deployment specific features. This is an area that’s currently under development and we’d expect to see more features lighting up in this area over time.
One feature that’s available today is the ability to add manual approval requirement to a deployment job. However, unlike in the classic pipeline where you’d create a manual approval requirement directly on the release pipeline, on a YAML pipeline you actually need to associated the deployment job with an environment and then add a manual approval on the environment.
You may have noticed that each of the deploy stages in the example YAML specified the environment_name property. This defines which environment you’re going to be deploying to. At this stage the only thing you can use this for in terms of deploying a mobile application is to require manual approval for the stage to continue. Let’s step through creating the environment and the approval requirement and you’ll see what I mean.
Create an Azure Pipelines Environment
Under the Pipelines tab, select Environments and then click the Create environment button in the center of the screen.
Next, provide a name for your environment and click Create. At this stage we don’t need to define any resources, so you can leave the default selection of “None”. The name that you specify for your environment has to match what you use as the environment_name property on the template.
Adding Manual Approval to the Environment
In order to add a manual approval requirement to an environment, simply open the environment (if you’ve just created the environment you’re already there). From the drop-down menu in the top-right corner, select Approvals and checks.
Next, click the + button in the top right corner.
Select Approvals, and then click Next
In the Approvals flyout you can specify a list of users and/or groups that need to approve a release to the environment. If you specify multiple users, each user needs to approve the release. If you specify a group, only one person in the group needs to approve the release.
In the Approvals flyout you can also specify a timeout; if the deployment isn’t approved for an environment within the timeout, the pipeline will fail.
Note: Currently there are no emails, or other notifications, sent to approvers. If you limit the timeout, once the specified period has elapsed if the environment hasn’t been approved, the pipeline fail and a notification will be sent out. The stage in the pipeline that failed can then be manually run and again, approval for the environment will be required.
Running the Build and Deploy Pipeline
Now when we run the pipeline, what we see in the portal is three different rows (one for each platform) with two stages (a build and deploy stage).
We’ve set the dependsOn element on each of the build stages to an empty array (ie ) meaning that they have no dependencies (btw the default is that stages will be done in the sequence that they appear in the YAML file, unless you indicate there should be no dependencies). Depending on how many build agents you have at your disposal the build stages may run in sequence, or in parallel.
Eventually, when each of the build stages completes, the corresponding deploy stage will light up indicating that it’s waiting for a check to be passed. There’s also a message box inserted into the interface to draw attention to the required approval.
Once all three of the build stages are complete, there are 3 approvals required; one for each of the deploy stages. The way we’ve structured the pipeline you have to approve the deployment for each platform.
If you wanted to require only a single approval for all three platforms you could inject an additional stage that was dependent on all three build stages. The approval would be required for this single stage, and then each of the subsequent deploy stages would be permitted to continue without further checks. The downside of this would be that you would have to wait for all builds to fail before you could deploy the app to any platform.
In this post we’ve looked at using a pipeline template for deploying Xamarin.Forms applications to AppCenter for testing across Windows (UWP), iOS and Android. In actual fact, and what I didn’t point out earlier, the deploy template can be used for any iOS, Android or Windows (UWP) app, not just a Xamarin.Forms application.
I also walked through setting up an environment so that you could add a manual approval step to the deployment process. Whilst the YAML pipelines don’t yet have all the features of the classic release pipeline, you can see from the way that the components connect and the UI that’s built in the portal that the foundations have been laid for future features to be built on.
One of the things I find frustrating is that for every new project we seem to have to recreate the build and release pipeline. In each case we step through the same steps, run into the same, albeit familiar, issues and end up with a pipeline that looks incredibly similar to the pipeline we setup for the last project we started. In this post I’m going to share the first set of Azure DevOps build templates that you can reuse in order to build your Xamarin Forms for iOS, Android and Windows.
This started out as an experiment to consolidate three different build pipelines into a single build pipeline. Often for mobile applications developers resort to setting up a build pipeline for each target platform, and then potentially each target environment. For us this because untenable as we were working on a white-labelled product that would need to have a build pipeline for each offering, which would grow over time.
The first step was to leverage the ability in Azure DevOps to create multi-stage pipelines. I setup a different stage for each target platform. You might be asking, why did I set them up as different stages; couldn’t I have just created additional tasks, or perhaps even different jobs for the different platforms. Well, the nice thing about different stages is that they can use different build agents. This is of course an absolute must, since iOS needs to be built by a Mac agent, whilst Windows (ie UWP) needs to be built by a Windows build agent. I could have cheated and wrapped the Android build with either the iOS or Windows build to cut down on the build time but you’ll see in a bit why I kept them separate.
In addition, my initial goal for this experiment was to have a standard build template that we could use and that I could share with the community via my blog. However, I felt this was only a part solution – there are so many great posts out there on how to setup a build process for XYZ but they become stale the minute they’re published. What if we could create a set of templates that could be released, much like a product, and could evolve over time.
In this post I’m introducing the Pipeline Templates github repository where I’ll be evolving build templates to help developers avoid the complexity of setting up the entire Azure DevOps, instead, focusing on building amazing applications. It’s still early days but check out the repository, watch and provide feedback so that we can evolve these templates as a community.
We’ll be starting with three templates that can be used to build a Xamarin.Forms application for iOS, Android and Windows. If we look at the basic structure of the repository, you’ll see that there’s a folder for Azure and a sub-folder for Mobile. The rationale was that over time this repository could house templates for other build platforms.
I also wanted to separate mobile app builds away from builds for the web or cloud services. However, the more I think about this the more I feel this separation is somewhat arbitrary because web apps built using SPA frameworks like React, Angular, Vue etc are similar in so many ways to mobile or desktop apps. I can imagine this structure will evolve over time – if you’re going to reference the templates in this repository, make sure you use the tags to ensure your build process doesn’t change or break as the templates evolve (more on this later).
Creating the Azure Build Pipeline
In order to use the pipeline templates you need to reference the GitHub repository as a resource within your build pipeline. I’ll walk through creating a new build pipeline but if you can add resources to an existing Yaml based pipeline using similar steps.
Start from the Pipelines tab in the Azure DevOps web portal and click the New Pipeline button (top right corner of the screen). Follow the steps to select your source code repository and pick a New pipeline template to use (I typically go with the Starter pipeline but either way we’re going to replace most of the yaml anyhow).
When you’re on the review tab, I suggest that you rename the yaml file. It’s not immediately obvious that you can change the name of the yaml file. The following image shows the default name on the left and then if you tap on the azure-pipelines.yml text you can actually edit it. On the right side of the image is the new filename and I’ve placed it in a pipelines folder to keep everything tidy in the repository.
Rather than making changes at the review tab, I just hit save (not the default “save and run” since we know the pipeline isn’t setup correctly).
Connecting to a GitHub Resource
According to the documentation you should be able to just add a GitHub resource directly to the Yaml file. However, in doing this I found that my pipeline didn’t work until I created an endpoint, similar to if you were to add other authenticated repositories as a resource.
Creating an endpoint is actually quite simple but you may well need additional permissions on your repository, depending on what access permissions you have. Click on Project Settings, in the bottom left corner of the Azure DevOps web portal, then click on the Service Connections tab. Click on the New service connection button, that’s in the top right corner of the Service connections screen. Select the GitHub connection type and then populate the New GitHub service connection flyout.
Once you click the Authorize button, you’ll most likely need to adjust the Service connection name – you’ll need this value in the next step.
The next step is to go back to your newly created pipeline (Note: if it doesn’t appear when you click on the Pipelines tab, you’ll need to switch view to All instead of Recent since you haven’t run your pipeline yet). Once you have the pipeline open, click Edit to bring up the yaml editor.
Replace the contents of the yaml file with the following:
This block of yaml references the pipeline_templates repository that’s owned by builttoroam. It’s going to use the Pipeline-Templates service connection to access the repository where it’s going to look at tag v0.1.0. In the pipeline this resource can be referenced as builttoroam_templates, which is just a local name assigned to this resource by the pipeline.
Important: that you specify the ref attribute and specify a tag. If you don’t, you’ll be pointing to the latest available templates on master. Referencing master is fine as you setup your pipeline but you should change to referencing a tag release to make sure that as we change the repository, your build continues to work.
Xamarin.Forms Build Pipeline
Now that we’ve added a reference to the pipeline templates repository, we can make use of any of the templates. For example the following yaml references the ios Xamarin template.
In this case to access the template in the pipeline templates repository, we firstly need to use the @[local resource name] syntax and then secondly we need to provide the path to the yml file (ie azure/mobile).
This yaml snippet defines the stages for the build pipeline, including a stage that’s defined in the referenced template file (in this case build-xamarin-ios.yml). You can add additional stages by simply repeating the -template section and adjusting the name of the template.
This code snippet shows just the first two parameters being passed in. At the end of this post is a full example showing the required parameters. There are some additional parameters that can be used to adjust output file name and folder, as well as for adding custom steps to the beginning, pre and post build and at the end of the stage.
The following is a fully worked example which includes iOS, Android and Windows. Note that there are a number of variables that are being passed into the templates. These are all defined within the Inspector XF Build Variables group, which can be defined via the Library tab in Azure DevOps.
There are also some secure files that have been added to the Library, which again are referenced here using a variable defined in the variable group. The process for adding secure files is that you upload the file, give it a friendly name and then assign that friendly name to a variable that’s in the variable group.
In this post I’ve given you a very quick introduction to the Pipeline Templates repository. Over the coming posts I’ll walk through some of the templates and what you can do with them in more detail. I’d love feedback on the templates – raise an issue to suggest changes on the GitHub repository and I’ll see if I can encorporate them.
Following yesterday’s post on repacking an Android APK, I was pointed to an awesome post by Daniel Causer entitled Build Once Release Everywhere – APK. Daniel’s post goes into a lot of detail on how to setup your build and release process. Specifically the scripts that he’s created for automating the repacking process I described in … Continue reading “Scripting the Repacking of Android Apps for a Release Pipeline”
Following yesterday’s post on repacking an Android APK, I was pointed to an awesome post by Daniel Causer entitled Build Once Release Everywhere – APK. Daniel’s post goes into a lot of detail on how to setup your build and release process. Specifically the scripts that he’s created for automating the repacking process I described in my post (you can grab them from his github repo.
I think the logical next step is for the creation of a Azure DevOps extension that either allows you to invoke apktool from a task in the build or release pipeline; or an extension that does the whole extract; replace/rename/modify file and then repack and sign. I suspect the former might be the best way as it would give developers the most flexibility. However, it would require more steps to be configured in the release pipeline.