Are you looking to boost your skills and knowledge in Database DevOps, learn how to transition to the cloud, and work across multiple databases?

Join me and fellow speakers/experts at three different locations across Australia (Brisbane, Sydney and Melbourne)to find out how your business can tackle the complex challenges of database management throughout the entire DevOps lifecycle, no matter what database you’re using or where it’s located. Hear from industry experts as we share our experiences on boosting efficiency, reducing errors, and protecting business-critical data.

This event is designed for database professionals who want to stay ahead in the fast-paced world of DevOps. Whether you’re an end-user looking to level up your skills or a tech enthusiast eager to dive into advanced solutions, the Redgate Seminar is your ticket to staying informed and empowered in the rapidly changing field of Database DevOps

A bit about me :

I am a Microsoft Data Platform MVP and VMware Expert with a passion for efficient database and application deployment using DevOps methodologies. I have 19 years’ of IT experience in managing and deploying large scale databases on SQL Server technologies.

I’ve been managing SQL Server since SQL Server 2000, and pragmatic approaches to delivering business value to clients is his career passion. Educating and helping others learn is a driver for me and I’m a PASS Chapter Leader, international speaker, and a repeat guest lecturer at a local university.

If you’re in any of these cities and this sounds like your thing, be sure to check it out and register with the links below!

Join us in Brisbane on 2nd May. Discover how your business can benefit from solving complex challenges of database management across the entire DevOps lifecycle on any database, anywhere. Hear from industry experts as they share experiences of how to increase efficiency, reduce errors and safeguard business-critical data.

Join us in Sydney on 7th May. Discover how your business can benefit from solving complex challenges of database management across the entire DevOps lifecycle on any database, anywhere. Hear from industry experts as they share experiences of how to increase efficiency, reduce errors and safeguard business-critical data.

Join us in Melbourne on 9th May. Discover how your business can benefit from solving complex challenges of database management across the entire DevOps lifecycle on any database, anywhere. Hear from industry experts as they share experiences of how to increase efficiency, reduce errors and safeguard business-critical data.

Hope to see you at one of these!!.

Yip.

This blog post is about Test Data Management – or my take on it.

I just saw that a video I did for my good mates at Redgate is just over 100,000 views.

Test data management (TDM) is all about handling the data used in software testing. It’s basically making sure you’ve got the right tools before starting a job – in this case, ensuring your test data is spot-on for checking if your software works as it should.

Now, in the world of DevOps, TDM is super important because it helps keep testing on track with the speedy development process. Pretty much meaning you need the right stuff to keep you going without slowing down.

How does TDM fit into DevOps?

1. Being Nimble: DevOps is all about being quick on your feet. TDM makes sure your testing can keep pace with development by giving you the right data when you need it. It’s like having your gear sorted so you can jump into action without missing a beat.
2. Making Things Easy: DevOps loves automation, and TDM is no different. It’s about setting things up so your testing practically runs itself. Just like having your chores automated so you can spend more time chilling out.
3. Setting Up Camp: DevOps likes to set up testing environments on the fly. TDM ensures these spots have everything they need for a good test run. It’s like having your camping gear ready to go whenever you feel like hitting the road.
4. Keeping Things Private: Test data often has sensitive info. TDM makes sure this stuff stays safe and sound during testing. It’s like keeping your personal info locked up tight so no one gets their hands on it.
5. Testing Early and Often: DevOps wants testing to happen early on. TDM helps make this possible by getting the test data ready ahead of time. It’s like doing your homework early so you can kick back and relax later.

So, in a nutshell, good TDM is the secret ingedient that keeps the DevOps machine running smoothly. It’s all about having the right stuff in the right place at the right time, so you can keep moving forward without missing a beat.

So for 2024 – what are you doing about managing your test data management?
Is it in the too hard column/basket? If so – don’t worry I am gonna be writing about this a lot…..

Yip.

A few weeks ago I did an interiview with Nagaraj Venkatesan – a good friend of mine who works for Microsoft Singapore.

You can watch video below – I talk about all things DevOps – where it relates to data, Microsoft Fabric and my thoughts on the impact of AI:
https://www.youtube.com/watch?v=jBseLdRH1FU

I hope you enjoy it as much as I did doing it.

Yip.

I love KQL so much I even made a video and if you compare it to my last blog post – yeah my hair has changed a bit…
And my beard.
And my clothing…

Check out Data Exposed here – it is an AWESOME site:

https://learn.microsoft.com/en-us/shows/data-exposed/

If you want to see a longer version of a session on KQL then do visit my last blog post here

I recently did a talk to the Data Platform Down Under User group and it covers a whole heap of stuff that I’ve written about in this series.

Check it out below:

I was talking to an ex-client earlier this week and he saw the drafts of my blog posts around DevOps infra-as-code spinning up Azure Data Explorer and he said – but what about ARM templates…..

Now this is a trigger. I do not like ARM templates.
But…

Some things pay the mortgage and back when we worked together I spun up their whole platform using ARM (this was before bicep was a thing).
So here it is for you Scott… you lover of ARM (and a good red wine).

Here are the steps to provision an Azure Data Explorer cluster using an ARM template.

Prerequisites:

• An Azure subscription
• Basic knowledge of Azure Resource Manager templates
• Being a lover of JSON (just kidding – I have to write this…)

Step 1: Create the ARM Template The first step is to create the ARM template. The template defines the resources that will be provisioned in Azure. We’ll create a simple template that provisions an Azure Data Explorer cluster.

Here’s an example ARM template that provisions an Azure Data Explorer cluster:

{
  "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "resources": [
    {
      "type": "Microsoft.Kusto/clusters",
      "apiVersion": "2021-04-01-preview",
      "name": "[variables('adxClusterName')]",
      "location": "[variables('location')]",
      "sku": {
        "name": "Standard_L16s",
        "tier": "Standard"
      },
      "properties": {
        "enableDiskEncryption": false,
        "dataRetentionTime": "365.00:00:00",
        "trustedExternalTenants": []
      }
    }
  ],
  "variables": {
    "adxClusterName": "adx-cluster",
    "location": "[resourceGroup().location]"
  }
}

This template provisions an Azure Data Explorer cluster with the name “adx-cluster” and the SKU “Standard_L16s” in the same location as the resource group.

Step 2: Deploy the ARM Template
To deploy the ARM template, we’ll use the Azure Portal. Navigate to the resource group where you want to provision the Azure Data Explorer cluster and click on “Deploy a custom template”.

Select the “Build your own template in the editor” option and paste the ARM template code into the editor. Click “Save” and then “Review + Create”.

Review the template parameters and click “Create” to deploy the Azure Data Explorer cluster.

Step 3: Verify the Deployment
After the deployment is complete, you can verify that the Azure Data Explorer cluster was provisioned successfully. You can check the status of the cluster in the Azure Portal or use the Azure CLI to run the following command:

az kusto cluster show --name adx-cluster --resource-group MyResourceGroup

This command will return information about the Azure Data Explorer cluster, including the status and SKU.

So there you have it – yes ARM templates are “relatively” easy, but they can be horrid..
Use bicep as a first go to or my favourite – terraform.

Either way – just use infrastructure as code to do any provisioning – OK??!!

(Also Scott – when I am in Seattle next – you own me a lot of red wine)

Yip.