Upgrade the Angular .NET Core SPA Template to Angular 9

Angular 9 has just been released and includes a number of major improvements. Be sure to check the official announcement to learn more. If like me you build single page apps with ASP.NET Core and Angular you will be keen to upgrade. This post details the steps required to upgrade the .NET Core project template to Angular 9.
Prerequisites
Ensure that you meet the following prerequisites:
Creating a new project
Create a Angular .NET Core SPA project from the command-line by running:

dotnet new angular –output AngularSpa –auth Individual

The ASP.NET Core project provides an API back end and the Angular CLI project (contained within ClientApp) provides the front end / SPA.

The output argument specifies the output directory name, and, since no name argument is specified, it is also the name for the new project.
Authentication can be added using the optional auth argument. Specifying Individual creates the project using ASP.NET Core Identity for authenticating and storing users combined with IdentityServer for implementing OIDC.

Build the new solution using the following commands:

cd AngularSpa
dotnet build

The above command restores NuGet / npm dependencies (takes a few minutes) and then builds the entire project.
Next, create a new git repository and commit the initial version using the following commands:

git init
git add .
git commit -m “Initial version”

In the next section, you will upgrade the project to Angular 9.
Upgrade to Angular 9
The process is relatively straightforward, aside from a couple of issues we will cover later. First, switch to the Angular SPA directory (ClientApp):

cd ClientApp

Next, run ng update to update to the latest 8.x version and then commit those changes:

ng update @angular/core@8 @angular/cli@8
git commit -a -m “Update to latest version of Angular 8”

Next, run ng update again to upgrade to Angular 9 and then commit those changes:

ng update @angular/core@9 @angular/cli@9
git commit -a -m “Upgrade to Angular 9”

If everything was successful, you will see the following message:

Your project has been updated to Angular version 9!
For more info, please see: https://v9.angular.io/guide/updating-to-version-9

Don’t get too excited, we are not done yet. You will receive the following error if you attempt to build the Angular project:

Generating ES5 bundles for differential loading…
An unhandled exception occurred: C:CodeScratchAngularSpaClientAppvendor-es2015.js: ‘with’ in strict mode (136749:4)

136747 | Window_run: function _run(code, file) {
136748 | if (file) code += ‘n//@ sourceURL=’ + file;
> 136749 | with(this) eval(code);
| ^
136750 | },
136751 | EventHandlerBuilder_build: function build() {
136752 | try {
See “C:UsersjasonAppDataLocalTempng-MEnshjangular-errors.log” for further details.

You can learn more about this issue here. However to resolve this issue, simply remove the following line from ClientApp/src/main.ts:

export { renderModule, renderModuleFactory } from ‘@angular/platform-server’;

Finally, run ng build and, if successful, commit your changes:

ng build
git commit -a -m “Resolved issue with template”

No further steps required! In the next section, you will launch the project to verify the upgrade was successful.
Launching the project
Normally launching the project is as easy as executing dotnet run, however, if you do so now, you will see the following error:
You can learn more about this issue here. Fortunately, it is easy to resolve this issue. First, open ClientApp/package.json and locate this line:

“start”: “ng serve”,

Then change to include an echo command as follows:

“start”: “echo Starting… && ng serve”,

Easy. Be sure to commit these changes too:

git commit -a -m “Resolved issue #2 with template”

Now launch the project from the root directory using dotnet run. The project will build, start the front and back end, and then display the following message:

Now listening on: https://localhost:5001

Browse to https://localhost:5001 to see the home page of the new project:
That’s it! You are ready to get started developing with Angular 9 and ASP.NET Core 3.1. Thanks for reading, be sure to take a look at some of the other posts in this category. If you did run into any issues, view the sample code for this post. If you can’t figure it out from there, please post a comment below.

Read More

Getting started with Form Recognizer (preview)

When I first heard about Form Recognizer, I thought to myself, “Wow, there are so many forms in the real world that needs to be parsed and digitilized!”. Once v2 preview came out, I finally decided to build an app that will parse through all of our user feedback from the last AI Hack Day.
Form Recognizer is an Azure Cognitive Services that allow us to parse text on forms in a structured format. Now we can extract the location and size (bounding box) for where information was entered or written along with the OCR’d text values.
For feedback forms this means, I can get feedback from users by merely uploading their scanned forms to trained custom Form Recognizer.
Prerequisite
In this blog post will attempt to recognize user feedback using Form Recognizer and Form Recognizer to do custom form recognition rather than built-in one.
We’ll need:
Filled out forms
Azure Account
Docker (optional)
Azure Storage Explorer
UPDATE 05/05/2020: The setup process is now automated and you’ll now be able to train and use custom forms much faster than before. 😀
1. Filled out forms
You’ll need at least 6 filled out forms for this demo.As co-organizer of AI Hack Day, I’m using the SSW User Group Evaluation Survey which you can download, fill out, and scan if you don’t have your forms.
Microsoft also have their own data set, if you prefer prefilled: https://github.com/Azure-Samples/cognitive-services-REST-api-samples/blob/master/curl/form-recognizer/sample_data.zip
2. Create Resources
To get started, you can either go to Azure Portal and create a new resource for Form Recognizer or use PowerShell with Azure CLI to create that resource faster. The script will save you about 15 minutes of work and works with existing resources (you can re-run it)!
Find the script on Gist: https://gist.github.com/jernejk/fdb42e032a9568d42dc8c2f05bd1fc13
Run with PowerShell Core (replace the […]):

.create-custom-form-recognizer.ps1 `
-FormRecognizerName [FormsName] `
-StorageAccountName [BlobStorageAccountName] `
-ResourceGroup [ResourceGroupName] `
-StorageAccountSasExpiry “2021-01-01T00:00Z” `
-InstallationType Web

Figure: Example run for Docker configuration.
Run set-executionpolicy unrestricted in admin mode if you don’t have permission to run scripts.
The script will:
Create Azure Blob Storage (name needs to be lowercase)
Correctly configure Azure Blob Storage (CORS and SAS)
Create Form Recognizer
Give you instructions and information required for completing the setup
You can decide how to run the tool by changing -InstallationType argument:
Website hosted by Microsoft -InstallationType Web (default)
Docker Container -InstallationType Docker
Running React app locally from GitHub source -InstallationType Web
If you’re interested how to configure everything from scratch, you can visit Train a Form Recognizer model with labels using the sample labeling tool at Microsoft Docs.
3. Create a project
Whew, that’s a lot of prep work for one service, but now we can start!
Run Form Recognizer Tool via Docker:

docker run -it -p 30000:80 mcr.microsoft.com/azure-cognitive-services/custom-form/labeltool eula=accept

Once running, go to “https://localhost:30000” and let us create a new project.
Add name
“Add Connection”
Add a name for the connection
Copy “Blob Storage SAS URI”

Folder path is empty in this case
Copy “Form Recognizer service uri” and “Form Recognizer API key” from the script results
We finally have our project created!

4. Start labeling
You’ll notice that the project is empty and it doesn’t give you options to upload images. To get your forms in (images or PDF), you’ll have to upload them to the Blob Storage we created for this project. You can use Microsoft Azure Storage Explorer to upload your forms.
TIP: Don’t upload all of your images because you want to leave some of them for testing.
Once they are uploaded, refresh the page on the Form Recognizer Tool.
Wait for OCR to complete
Create tags in the right column you want to recognize (name, email, etc.)
Select the detected text (it will be light green)
Select the correct tag (e.g. “Position”)
The selected text will now have the same color as the tag
Do this for all of the text you want to be tagged on all of the forms.
NOTE: This tool supports only one form per image/PDF. Form with multiple pages will be treated as a single multipage form!
5. Training
Go to Train icon
Click “Train”
Done! 🎉

6. Testing
Go to 2 squares icon
Browse
Predict! 🎉

NOTE: If using are free tier, this may take a while because of the rate limit.
Personally, I’m using PowerAutomate to use Custom Form Recognizer because I can add Form Recognizer as a step in flow with no code. Instead of writting an app, I simply create a button trigger in flow and share that flow with other admins, which allows them to use it on their phone or desktop.
What’s next
The form recognizer works mostly well however, there are a few issues I need to address:
OCR isn’t always great especially if someone’s handwriting isn’t great
This version doesn’t recognize checkboxes (the feature is on their backlog)
When uploading a multipage PDF, it treats it as a single form on multiple pages. We need to split it up into multiple images/PDFs
At the moment I’m experimenting with recognizing checkboxes myself. However, I’m happy that we can quickly get images into structured data and do a bit of correction and add ratings rather than typing everything from scratch. I’m looking forward to the day when our feedback forms for hack days and user groups are automated and we can send feedback to guest speakers much quicker (and enjoying the weekend instead of entering data! 😁).

Read More

Introducing SSW People!

About a year ago, during SSW’s annual retreat in Brisbane, we started to imagine what the world would look like if the SSW Rules site was easier to use.  Therefore, we had to find an alternative to SharePoint’s content management features.The first tech we evaluated was an Angular application. Even though we love building complicated applications on Angular, we discounted it as it didn’t suit simple content pages. Next we investigated static site generators and realized this was going to be much better.
Going JAMStack
There are a lot of static site generators such as Jekyll, NextJS, and Gatsby all with strengths and weaknesses. Collectively they are known as the JAMStack

The Jamstack is not about specific technologies. It’s a new way of building websites and apps that delivers better performance, higher security, lower cost of scaling, and a better developer experience.
Pre-rendered sites can be enhanced with JavaScript and the growing capabilities of browsers and services available via APIs.
https://jamstack.org
Dipping our toe in something smaller
We also quickly realized that the old SSW Employee pages (was https://sharepoint.ssw.com.au/AboutUs/Employees) were very similar to our SSW Rules pages, being:
Static content
Hosted on SharePoint
Publicly accessible
So we built the Employee Pages first – basically a prototype for Rules V2. By that time we were approaching NDC 2019 and got busy on various other projects, such as the SSW Rewards App… however on SSW bench time Matt W. and Jean T. were able to continue it and released it January 2020. 
Of course, things got a bit more complicated than initially planned, and we had to deal with a few hiccups – such as changing our preferred JAMStack and migrating SharePoint history into GitHub history. Thank you also to Tiago, Harry, Sebastien, and David.
Figure: Index – Old site on SharePoint Figure: Profile – Old site on SharePoint
Announcing SSW People
I am happy to announce the release of our SSW Employee Pages V2, now re-branded as SSW People – https://www.ssw.com.au/people
Figure: The new Flow (without SharePoint) Figure: Index – New site powered by GitHub and JAMStack Figure: Profile – New site powered by GitHub and JAMStack
A month ago we released it internally
Everyone’s old profile (i.e. SharePoint) was locked and put in a read-only state. Then the data was migrated from SharePoint to GitHub… and soon there’ll be one less SharePoint server.
The changed workflow was well received.
Employees now have their instructions out in the open to edit their profile: https://github.com/SSWConsulting/People/blob/master/README.md
Every time they edit their GitHub profile page the site pages are re-generated.
Every time they edit their GitHub profile, they’re getting more dots on their GitHub stats page
Let’s look at Jean’s page as an example: https://www.ssw.com.au/people/Jean-Thirion – the “edit” link takes you to: https://github.com/SSWConsulting/People/blob/master/Jean-Thirion/Jean-Thirion.md
Then we got feedback on Microsoft Forms
TODO: [Include stats from the MS Form]
The Tech Stack
Frontend – JAMStack, Gatsby – https://rules.ssw.com.au/static-site-generator 
Note: We evaluated a number of frameworks, including Jekyll (it was great, but too simple) and Next.js (we had reliability problems with page generation), and then settled on Gatsby (works natively with GitHub as a data source and you can query via GraphQL based on tags and branches).
Data format – Markdown – https://rules.ssw.com.au/using-markdown-to-store-your-content
Data – GitHub 
Data – Dynamics 365 skills
Source Code – GitHub 
Pipelines – Azure DevOps
The GitHub repo is available here for those who want to have a closer look: https://github.com/SSWConsulting/people.ssw.com.au
This is a Gatsby generated website, leveraging data from SSW Profiles (now on GitHub!) https://github.com/SSWConsulting/People and our CRM (thanks to William for the integration). Pages are automatically re-generated and published to our SSW website when a profile markdown page is updated. 
Performance – SharePoint vs Static Pages
Moving to static pages generated with Gatsby provided a significant performance boost. Specifically from 68% to 100%
Figure: Before – Adam’s page performance – No throttling Figure: After – Adam’s page performance – No throttling Figure: Before – Index page performance Figure: After – Index page performance
Not only do the pages load heaps faster, they *feel* much more responsive.
Redirects
As we changed all profile URLs, we added 301 redirects from all the old pages to the new ones (see Do you use “301” code to redirect renamed or moved pages?). This ensures we don’t lose our Google juice and users wont see 404’s.
What we don’t have
Location – subdomains vs subdirectories
Which one do you like better? A subdomain (people.ssw.com.au)​ or a subdirectory (ssw.com.au/people)?We decided to use subdirectories because of the nice URL. I was worried about SEO impacts, but after a fair bit of research we discovered that from a purely SEO ranking perspective, the ‘sub folder versus sub domain’ debate is a bit of a myth – it doesn’t really matter. Google is smart enough to work it out. So it really comes down to personal preference.
Summary
As usual, I am very keen to get your feedback. Please have a look and click around. Send any changes to me, or even better be modern and use GitHub Issues (via the edit button in the top right-hand corner).
Now that SSW.People is live, we will likely attack the big monster that is SSW Rules V2. We may follow the exact same process that we used for SSW.People and hopefully it will go much quicker this time, thanks to the lessons learnt. New challenges ahead! Stay tuned

Read More

Clean Architecture with .NET Core: Getting Started

Over the past two years, I’ve travelled the world teaching programmers how to build enterprise applications using Clean Architecture with .NET Core. I started by providing a sample solution using the iconic Northwind Traders database. Recently, I’ve developed a new Clean Architecture Solution Template for .NET Core.
This post provides an overview of Clean Architecture and introduces the new Clean Architecture Solution Template, a .NET Core Project template for building applications based on Angular, ASP.NET Core 3.1, and Clean Architecture.
Let’s start with an overview of Clean Architecture.
Overview
With Clean Architecture, the Domain and Application layers are at the centre of the design. This is known as the Core of the system.
The Domain layer contains enterprise logic and types and the Application layer contains business logic and types. The difference is that enterprise logic could be shared across many systems, whereas the business logic will typically only be used within this system.
Core should not be dependent on data access and other infrastructure concerns so those dependencies are inverted. This is achieved by adding interfaces or abstractions within Core that are implemented by layers outside of Core. For example, if you wanted to implement the Repository pattern you would do so by adding an interface within Core and adding the implementation within Infrastructure.
All dependencies flow inwards and Core has no dependency on any other layer. Infrastructure and Presentation depend on Core, but not on one another.
Figure: Clean Architecture Diagram
This results in architecture and design that is:
Independent of frameworks it does not require the existence of some tool or framework
Testable easy to test – Core has no dependencies on anything external, so writing automated tests is much easier
Independent of UI logic is kept out of the UI so it is easy to change to another technology – right now you might be using Angular, soon Vue, eventually Blazor!
Independent of the database data-access concerns are cleanly separated so moving from SQL Server to CosmosDB or otherwise is trivial
Independent of anything external in fact, Core is completely isolated from the outside world – the difference between a system that will last 3 years, and one that will last 20 years
In the above design, there are only three circles, you may need more. Think of this as a starting point. Just remember to keep all dependencies pointing inwards.
Let’s take a look at a simple approach to getting started with the new Clean Architecture Solution Template.
Solution template
This template provides an awesome approach to building solutions based on ASP.NET Core 3.1 and Angular 8 that follow the principles of Clean Architecture. If Angular is not your thing, worry not, you can remove it with ease. In this section, you will install the template, create a new solution, and review the generated code.
Prerequisites
The first step is to ensure you meet the following prerequisites:
Check the .NET Core version by running this command:

dotnet –list-sdks

Check the node version by running this command:

node -v

Next, install the solution template using this command:

dotnet new –install Clean.Architecture.Solution.Template

Create a new solution
Creating a new solution is easy. Within an empty folder, run the following command:

dotnet new ca-sln

The following message will be displayed:

The template “Clean Architecture Solution” was created successfully.

This command will create a new solution, automatically namespaced using the name of the parent folder. For example, if the parent folder is named Northwind, then the solution will be named Northwind.sln, and the default namespace will be Northwind.
The solution is built using the Angular project template with ASP.NET Core. The ASP.NET Core project provides an API back end and the Angular CLI project provides the UI.

Launching the solution from Visual Studio 2019 is trivial, just press F5.
In order to launch the solution using the .NET Core CLI, a few more steps are required. You can learn more by visiting the above link, but I’ll include the information here for completeness.
First, you will need an environment variable named ASPNETCORE_Environment with a value of Development. On Windows, run SET ASPNETCORE_Environment=Development. On Linux or macOS, run export ASPNETCORE_Environment=Development.
Next, run the following command from the solution folder:

cd src/WebUI
dotnet build

NoteThe initial build will take a few minutes, as it will also install required client-side packages. Subsequent builds will be much quicker.

Then run dotnet run to start the application. The following message will be displayed:

Now listening on: https://localhost:port

The port is usually 5001. Open the web site by navigating to https://localhost:port.

NoteYou will also see a message similar to the following:NG Live Development Server is listening on localhost:port, open your browser on https://localhost:portIgnore this message, it’s not the URL for the combined ASP.NET Core and Angular CLI application

If everything was successful you will see the following:
Let’s take a look at the structure of the newly generated solution.
Solution structure
The solution template generates a multi-project solution. For a solution named Northwind, the following folder structure is created:
The project names within src align closely to the layers of the Clean Architecture diagram, the only exception being WebUI, representing the Presentation layer.
The Domain project represents the Domain layer and contains enterprise or domain logic and includes entities, enums, exceptions, interfaces, types and logic specific to the domain layer. This layer has no dependencies on anything external.
The Application project represents the Application layer and contains all business logic. This project implements CQRS (Command Query Responsibility Segregation), with each business use case represented by a single command or query. This layer is dependent on the Domain layer but has no dependencies on any other layer or project. This layer defines interfaces that are implemented by outside layers. For example, if the application needs to access a notification service, a new interface would be added to the Application and the implementation would be created within Infrastructure.
The Infrastructure project represents the Infrastructure layer and contains classes for accessing external resources such as file systems, web services, SMTP, and so on. These classes should be based on interfaces defined within the Application layer.
The WebUI project represents the Presentation layer. This project is a SPA (single page app) based on Angular 8 and ASP.NET Core. This layer depends on both the Application and Infrastructure layers. Please note the dependency on Infrastructure is only to support dependency injection. Therefore Startup.cs should include the only reference to Infrastructure.
Tests
The tests folder contains numerous unit and integration tests projects to help get you up and running quickly. The details of these projects will be explored in a follow-up post. In the meantime, feel free to explore and ask any questions below.
Technologies
Aside from .NET Core, numerous technologies are used within this solution including:
In follow-up posts, I’ll include additional details on how the above technologies are used within the solution.
Additional resources
In this post, I have provided an overview of Clean Architecture and the new solution template. If you would like to learn more about any of these topics, take a look at the following resources:
Thanks for reading. Please post any questions or comments below.

Read More

Fires + Adam’s 2019 in Review

2019 ended in uncertainty for a lot of people across Australia. Our bush fire season started a lot earlier than usual, and this year the fires are insanely bigger, and burning more land than ever before. It’s been devastating, and there is no end in sight for some areas that have been affected. What has stood out, is the incredible leadership from Shane Fitzsimmons, the Commissioner of the New South Wales Rural Fire Service – he has been compassionate and leading from the front and a good communicator. He’s been the face of the fires.It’s the main topic of conversation here, and all Aussies are grateful for the commitment and dedication from our poor tired Rural Fire Service volunteers. Some of them have been fighting fires now for over 100 days – an incredible feat. It’s purely their love for their communities keeping them going through the exhaustion and financial stress. I don’t think we’ve ever wished more for rain.
Like many Aussies, I have received so many concerned emails from people around the world. I do hope the rest of the world sees this event as the canary in the coal mine, and what’s going to happen elsewhere.
So many families have lost homes – around 2,000 houses have been lost, and when you see people’s reactions to that news, it’s really heartbreaking.  It affected many people I know, and our quality of life is quite a bit lower – smog, filth, ash in the water supply, even black water at the beach!
The amount of current financial devastation Australians are going through today, will pale in comparison to when we see the long-term impacts.
[embedded content]
Video: Look at the water at my local Maroubra Beach!
Figure: City slickers are a little removed, other than the ash that is in the air and filthy cars. Look at my car a day after having it washed – this is a visual image of what we are currently breathing everyday
Figure: In remote areas, it is a lot more serious. This is Jennifer Haley’s property in Batlow, NSW. Her family lost everything on their property except the house. There is no washing this car!
I think it’s important in times like this, to remember the things that we are thankful for. Like many, for me it’s my family. My beautiful girls and my team at SSW. My eldest daughter Eve just finished her HSC with some awesome results, Ruby continues to be outgoing and headstrong and she’s about to start driving, which will give me a heart attack.
I’m also really proud of everything that my team at SSW have accomplished this year. We have increased in size as a company, and we have produced some quality web applications for our clients.
Here is an overview of our year:
Figure: SSW’s 2019 in Review – 2020 here we come!
SSW’s 2019 Projects
I love all of my Power BI Reports, I see facts and sexy visualisations for just about everything SSW does.
According to Power BI, in 2019 SSW completed and deployed 152 client projects, which is huge as each one of those was successfully deployed (some hundreds of times).
Figure: The big green slice represents 70%, showing that JavaScript frameworks remain king! Interestingly we also had an increase in Dynamics and CRM work (the black slice), keeping our resident Dynamics 365 gurus very happy (and busy)! Figure: Breaking down that 70%, we see of all the web work (JS Frameworks, .NET Core and CMS). In the last year SSW’s biggest slice of the pie was Angular development, and for the first time, .NET Core has overtaken our MVC work.
Standout Projects:
The projects that I’m most proud of this year are:
Hutchison Weller – see video for KNOWnoise was re-developed as a web application and deployed within 3 months. It allows Hutchison Weller’s clients to accurately plan the noise outputs for upcoming construction projects. This application was built using Angular and NgRx in the frontend. The backend was developed using .Net Core, hosted on Azure, and following the Clean Architecture principles.
Sydney Uni – see video for BREAST is a web application that allows radiologists to continuously practice their cancer diagnosis in a safe environment using real case studies. It was built in .NET Core for the backend and Angular for the front-end and the whole application was hosted in Azure.
SSW’s Reward App – this is a mobile app we built for the community to use that we launched at NDC Sydney. It is now our main way of engaging with the dev community, using it for communication, user group prizes and more!
SSW TV
Each year our video team does better and better work. Our YouTube channel is a real success. We’ve added some great team members to the SSW TV team this year so that our Multimedia expert Raj Dhatt has the support he needs to make great content.
There’s a bunch of great technical video resources including our user group events, the conferences we’re involved with, interviews with experts, and more. I’m really proud of it.
The analytics for SSW TV on YouTube, show that 2019 was our strongest year yet! In the past 12 months, SSW TV:
Released 44 new videos to the public
Acquired 6,087 new subscribers
Received more than 0.5 million views
Was watched for 3,344,580 minutes (that’s 6 years and 132 days!)
Our most popular video released in 2019 was ‘Clean Code with Entity Framework Core’ by Brendan Richards, which accounted for 8% of new views in 2019. And our most popular video of all time is now ‘Clean Architecture with ASP.NET Core 2.1’ by Jason Taylor, which was released in October 2018. It now has 246,887 views and it’s over an hour long!
All up, we’re at:
547 videos
28,184 subscribers
Almost 3 million views – wow!
19,338,480 minutes watched (that’s 36 years and 134 days!)
Figure: Our subscribers are slowly growing every year
Conferences and community engagement
Presenting is one of the favourite parts of what many SSW developers do, and I’m always on the lookout to work out what stuff we should share to make us, and the industry better.
Figure: My ‘Rockstar’ moment at my favourite conference NDC Oslo.
For me, my top 10 events in 2019 were:
My favourite talks were:
A couple of other really fun community activities I was involved with included:
Microsoft Ignite & meeting Satya Nadella who was the keynote speaker
HostingTroy Hunt at the Reactor in Sydney, where I got to introduce him to a crowd of 300 keen devs interested in security
SSW’s Angular and Xamarin Hack Days. These events are always great for meeting new devs and working with them to learn new tech.
SSW TV’s ‘Tech Tips’ at NDC Sydney, where I got to interview a few of the other conference speakers and got them to do a tech demo with me.
SSW Training
The event stats for 2019 were:
34 paid events (many sold out!)
45 free events
Figure: SSW’s New Course on Clean Architecture with Jason Taylor
In 2020 we’re going to be running updated 1-day and 2-day developer training courses in areas such as Azure, Angular, React and .NET Core. In addition, with the growing importance of clean development, we have launched a new course on Clean Architecture due to run in February 2020. It’s led by SSW’s Jason Taylor, who’s video Clean Architecture with ASP.NET Core 2.1 has attracted close to 250,000 views on YouTube. That will be an exciting event.

Read More

Retrospective: 2019

Inspired by my friend’s retrospective blog post, I have also decided to make a retrospective. For me, a retro isn’t just about looking at the past and fixing old mistakes, but also to figure out what works and celebrate achievements. You can read my friend’s blog post for a better description of what a retrospective should be and I’ll dive straight into it. 😀
Overall, 2019 felt very short and professionally, I have experienced more growth than in a typical year.
What went well
I presented in soooo many conferences and user groups… 24 in total!
AI Hack Day finally got a green light from my boss and is coming early this (2020) year in Melbourne, Brisbane, and Sydney!
My blog is reaching about 3k views and 900 users per month!
The feedback on presentations is fantastic! For some, my talks are one of the best talks in the conference (DDD, NDC Sydney, Global AI Bootcamp); I’m very humbled to hear that 😁
I found that my talks about ML .NET got a real impact on devs and got me invested into AI even more than I was before
I lived in Melbourne for about 3 months and it was awesome!
Jumpstarted an open-source project Cognitive Studio for exploring Cognitive Services
I traveled a lot and I enjoyed it! I even became a Qantas Silver member for almost free due client work travel
I have been recognized as a speaker and professional by many people. I’m now known for my Real-time facial recognition and ML .NET talks across Australia
Despite imposter syndrome, I realize that the knowledge that I share with others is accepted even by professionals that know the subject significantly better than myself
People like to talk to me after I have delivered my talks and I don’t have to stumble around because of my awkward social skills 🤣
My brother visited me in Melbourne!
Witcher TV series was the best things that happened on TV in 2019, topping Avenger’s Endgame and Rick and Morty
What didn’t go so well
I gained weight (like my friend 😂)
I have traveled so much this year that I want to have a break from travel for a bit (6 months of non-stop travels)
I’m spending too much time on YouTube watching random stuff
Getting my permanent Visa is significantly slower than expected
I still need a lot of time to prepare for talks, even if it’s an existing one. I always try to improve the talk and I put a lot of stress on my self to make sure I don’t mess up the talk
Imposter syndrome is stronger than ever, especially when I talk about ML .NET
I feel like the first half of the year was wasted on YouTube and not doing much. Most of the things I’m proud of started to happen around the middle of the year
I still didn’t upgrade my blog website even though it’s long overdue
What to improve
Lose weight
Travel outside Australia
Have more holidays and relax a little bit
Publish 1-2 blog posts per month
Have 12+ talks in a year
Contribute to the community more effectively
Take care of myself and spend more time with people I care about
Improve social skills with strangers as some know, they aren’t great 🤣
Final thoughts
Last year was an interesting one from many perspectives. I got into my 30s, I became a presenter, lived in a different city for a couple of months, got more positive feedback in one month for several months than I usually get in a year, etc.
On top of that, it charted a new direction for me, teaching devs how they can use AI even if they are not very familiar with the technology. It felt very humbling and rewarding when seeing private messages on Twitter and LinkedIn, people saying they managed to solve their problems because of my talks. As such, William Liebenberg and I managed to create a free event AI Hack Day backed by SSW and more may come before the end of the year.
On the non-professional side, I’m trying to exercise more, improve my diet and trying to find something I would love outside tech so that I’m not glued to screens 24/7. 😁
In short, this year might be the busiest so far and I can’t wait to get started!

Read More

8 Tips to Strengthen Your DevOps (Project KNOWnoise)

[embedded content].NET, NgRx and Angular – How We Developed ‘KNOWnoise’ with Hutchison Weller
At SSW each year we are lucky to deliver a lot of cool projects. I want to call out one project that solved a complex business challenge, and was fun to deliver. It had the usual, modern web technologies and cloud, and the project really benefited from great DevOps practices. The lead developer was SSW Solution Architect, Matt Wicks. Matt is also a Microsoft DevOps FastTrack Partner Consultant, and he runs a great team, as well as being passionate about great DevOps. 
The Client was Hutchison Weller and they had a vibrant Product Owner, plus a cool project name. Project KNOWnoise is an application that enables them to know in advance, the construction noise that machinery will generate.
MAKING THE CLIENT HAPPY 
Hutchison Weller had a unique requirement, they needed something that could help manage the environmental impacts their clients face when planning and delivering construction projects. If their client had a construction project that was going to create a lot of noise, they were going to have to pay a lot of $ for accommodation. If the app could predict who was going to be affected and how badly, then it could help to save a lot of $ for that client. 
The major part of the project was to help manage the noise outputs created by machinery and construction, and the effect they have on the nearby public. The developed application, KNOWnoise™ enables a DIY form of noise assessment.
BUILDING THE WEB APP 
Like too many applications, the client’s initial modelling for this was developed in Excel using a series of very complicated Lookups to do the calculations. While it worked on smaller projects, it couldn’t handle large construction projects as these could have large construction areas and potentially thousands of buildings where the construction noise would be heard – this would multiply out into millions of rows in Excel (which it couldn’t handle).  
Poor old Excel was getting a bad rap as they were encountering many other technical issues including: 
Excel would freeze regularly because of the way it was doing calculations 
If you were using a different version of Excel the macros might not work 
Excel had size limitations so they couldn’t use it on the large projects 
The new web app was built using: 
Angular, NgRx, and .NET Core  
Hosted in Azure 
Scrum 
Architecture – Built using Clean Architecture principals – watch Jason Taylor’s awesome video: Clean Architecture on the topic and SSW’s Rules to Better Clean Architecture.  
Automation – with Azure DevOps build and release pipelines. Regular automatic deployments to dev, test and production environments 
Quality – Extensive unit and integration testing 
Quality – Every pull request automatically ran the tests (great DevOps)
An awesome software team! 
This gave the team the confidence to do fear free refactoring when moving to the scary “large” projects 
Figure:  On the left of the screenshot it shows the different machinery and the project noise levels. On the right it shows the affected properties. Red is very bad, yellow is noisy, green is good.
Figure:  In this screenshot the red indicates a large cluster of houses that will be seriously affected by noise. If there is night time work, then the occupants would need accommodation.
MOVING ONTO THE SCARY “LARGE” PROJECTS 
I’m a fan of “if you’re going to fail, then fail fast”. So before launch, the first project we tested, was one that was much larger than the client had ever previously worked on and it performed great.Before each new project, a data import step needs to be undertaken. This posed an issue as there would be a lot of data (in some cases 2+ million rows) to import and validate before a project can be used by end users. Inserting that into Azure SQL requires a lot of $ DTUs (the Azure charge unit for memory and CPU). We didn’t want to put in any manual steps, e.g to dial it up to improve performance. The problem would be, if they forgot to dial it back down, that could lead to a $25k Azure bill for the month. WOOOOAH!
Note: Cosmos DB would have been faster, but would have cost a lot more. So it wasn’t really an option due to the volume of data. The data imports are not a frequent activity so Azure SQL database was chosen.
WHAT WAS THE SOLUTION? 
There were a few ways SSW managed to revolutionise the app. They pulled all of the lookup data and calculated results out of the SQL database and stored them as JSON files in Azure Blob Storage. This eliminated the need to hit up Azure SQL for thousands of reads, updates and deletes every time that a user would make changes to their construction plans; this sped up user operations and reduced the number of DTUs that the database required for normal daily use. Azure Blob storage is a fraction of a cent per GB, so from a data volume perspective it is much cheaper. Importing data into database and reading and saving results was faster, it had a more predictable response time, and avoided worrying about noisy neighbours in a database, and since a lot of data was no longer in the database – it shrank drastically. More importantly, its cost to run reduced drastically. 
In order to make this happen, we did a big refactor of the code, but since we were using the Clean Architecture pattern we knew where to make the changes, and we had all of the unit tests and integration tests in place meaning we had a high level of confidence that we could make those changes. Great DevOps pays for itself!  
INVESTING IN GOOD DEVOPS 
Investing in good DevOps practices from the beginning pays for itself very quickly. Having your unit and integration tests in place means you can make sure that you are reducing the risk of introducing regressions. Running your tests with every Pull Request means you can move forward faster and with more confidence as your master branch will only contain a stream of clean commits. If someone needs to take over the project, or picks up a new piece of work, they will start from a clean starting point. The process picks up bugs early.  
HYPOTHESIS DRIVEN DEVELOPMENT 
While we wouldn’t always recommend failing, it had its place in this project. We decided to use hypothesis driven development to see if we could discover the best architecture model to solve our data import issue. We had logs and stats from Application Insights to discover the root cause of our problem. We had a few ideas on how to solve it, so we developed and tested a few options to solve the problem, for example, we tried Azure Functions, then we tried Cosmos DB, before finally testing Azure Blob Storage. We had the stats for each of the runs and were able to empirically compare the results to determine that Azure Blob Storage was the best solution for this problem – it was the simplest, it provided the best performance and turned out to also be the cheapest. It just goes to show how important it is to keep an open mind in terms of storage; not everything needs to be in a SQL database. 
OTHER COOL STUFF   
We were able to do some cool things in Google Maps. Half of the interface uses Google Maps and we were able to overlay some cool visualisations to help people do their job. We were really happy that Hutchison Weller was able to use the extensible system we developed and changed the way users select where construction was taking place – they were able to change the overlays on the map from squares to hexagons without our involvement. This meant that they can choose where construction work is being done with greater accuracy. Visually it’s a lot cooler.  
Regarding the Azure cost, we were blown away with how cheap it is to run to this app, considering where it started from, and where it is now. We’ve improved performance, past the spec. The cost of running the app is a drop in the ocean, which is a huge win for Hutchison Weller. This app has allowed them to land some very large projects, like West Connex and other large infrastructure projects affecting millions of people. A huge win for a smaller company.  
8 Tips to Strengthen Your DevOps
Testing – automate your tests and run them before you start work and when you finish your work – if there is a problem, fix it straight away 
Pull Requests – they should trigger builds that run all your tests. Remember tests that aren’t executed, atrophy quickly
Deploy – use Azure and deploy to it regularly. Run your integrations tests against it. Running your application on Azure is very different to running it on your local machine. The earlier you discover these differences the better
Automate – have a release pipeline in place with environment promotion, don’t deploy to production unless you have successful deployments to your staging environment.  
Monitoring – use Application Insights and logging, this will help with postmortems and debugging production 
Source Code – merge your code frequently, this will reduce your merge and integration debt 
Deploy – release each PBIs as completed. This can be part of your definition of done. It will help you discover issues very quickly – rollback is easier and it’s easy to isolate the problem area
Deploy – keep your configuration settings and secrets out of code. Use KeyVault in your ARM templates
Having trouble with Clean Architecture? We have some great resources here: https://rules.ssw.com.au/the-best-clean-architecture-learning-resources 
Some projects feel like cross country skiing – hard work! This project felt more like down hill skiing – the pull requests, the clean architecture and the good DevOps helped heaps. SSW solved the client’s complex business problem and moved their business from a fragile Excel solution (with GOTO statements) to a long term solution running on modern web technology and Azure. They now have a solution that will grow with them over the years.
I’m so proud of this project and I hope you enjoy the case study.

Read More

Interview – Machine Learning that's ACTUALLY easy with Richard Campbell

Do you want to try machine learning, but don’t want to invest too much time learning a new programming language or some other complicated API or complicated online tools?
Richard Campbell and I are geeking out about why you would want to look at machine learning, how to get started, and what should you know when building your prototype. 🧐
ML.NET Model Builder opens up the world of Machine Learning to all developers.
[embedded content]
I had a blast doing the interview. I hope you’ll enjoy watching it as much as I enjoyed making it! 😁
Updated demo (screenshots)
Here are updated screenshots for ML .NET v1.4 Model Builder.
Figure: Start ML.NET Model Builder Wizzard.
Figure: Select Issue Classification.
Figure: Select file, column to predict (label) and input columns used for prediction.
Figure: Start training.
Figure: See results of training.
Figure: You can try out your model before generating the code.
Figure: Generate the code.
Figure: It generates 2 projects. ConsoleApp is testing purposes and Model project is the one that you can use in your apps.
Figure: In ConsoleApp you can see the usage of ML Model.
Bonus
When using ML .NET Model Builder, it will try to determine if a column should be a hash table or text, which can have a significant impact on your use case. It decides based on the ration of unique and duplicate values and it might choose the wrong type because of that.
However, after generating the code, you can change from a hash table to text and vice versa.
In file ModelBuilder.cs in method BuildTrainingPipeline change the .Append for desired column.
From:

.Append(mlContext.Transforms.Categorical.OneHotHashEncoding(new[] { new InputOutputColumnPair(“Description”, “Description”) }))

To:

.Append(mlContext.Transforms.Text.FeaturizeText(inputColumnName: “Description”, outputColumnName: “Description”))

Figure: Update the Append from Categorical hash to Featurize text.
Retrain model by running ModelBuilder.CreateModel(); in Program.cs. Return right after training because the model hasn’t been copied yet to bin folder.
Figure: Retrain model with feature text instead of categorical hash.
Remove the code from step 2.
NOTE: If your project contains . in it’s name, chances are that the path to the model will be incorrect. (a bug since v1.3) Make sure that path MODEL_FILEPATH in ModelBuilder.cs is correct!
Figure: Model Builder has a bug in generating the right path if project contains a dot in the name.

Read More

SSW had fun at NDC – gotta catch ’em all!

The NDC conference gets better and better every year. I look forward to it, and the SSW devs say it’s awesome. It’s located at the Hilton in Sydney’s CBD, and runs over 5 days. I attend a lot of conferences around the world and NDC is my favourite developer conference. The first 2 days are […]

Read More

Publish client-side Blazor to GitHub pages

Lately, I got fascinated by client-side Blazor, as I can build and host a .NET Core application with $0 cost! This is because Blazor only needs to be hosted somewhere on the web and does not require server-side logic.
However, if you try to publish Blazor to GitHub pages, you’ll notice it doesn’t quite work and the official MS documentation is way out of date. I’ll assume you have already created your GitHub Page.
1. Copy files from official Blazor demo
You’ll need to copy 404.html and .nojekyll from Official Blazor demo (yes, almost 2 years ago).
2. Replace content in index.html
In index.html replace your

Read More