Crafting Data - Augmenting results from Azure DevOps build/test pipelines

To understand the health of the code in QUBO alogrithm, we needed to understand the performance of the algorithm. We ran a sanity test on each checkin and then a more comprehensive test suite each night there was a checkin. The reporting provided Azure DevOps Analytics for build / tests provided was not enough to evalute the alogrithm performance, so we augmented the information from the tests and created this report. information from a test runner point of view.

Contacter le partenaire Voir la vidéo
Why did we create this?

To understand the health of the code in QUBO algorithm, we needed to understand the
performance of the algorithm. We ran a sanity test on each checkin and then a more
comprehensive test suite each night there was a checkin. The reporting provided
Azure DevOps Analytics for build / tests provided was not enough to evaluate the
algorithm performance, so we augmented the information from the tests and created
this report.

  • What questions are answered?
  • What is the health of the tests for each pipeline ?
  • For each test, how long did the algorithm take to run ?
  • What is the trend for each test run ?
  • Did the algorithm find the correct answer for this test ?
  • If a test failed, was it because of a incorrect result or from maximum time allowed for the test ?

How was this implemented?

Each test is an execution of an algorithm with a given testset. The custom data captured is
the input to the algorithm, amount of time the algorithm is allowed to run, actual run time
for the algorithm, and the result provided by the algorithm.

This custom data is written an Azure Blob in .csv format. An Azure Pipeline then pushes the
test run information (.trx) into to an Azure blob. The files in the Azure Blob are organized
by the build id and the name of the pipeline. 

Finally, the .pbix on each refresh reads the files in the Azure Blob into a semantic model
and the updates the reports and dashboards.

What next?
  • What is the environment for each test (cpu, memory, gpu characteristics) ?
  • Integrate Azure Devops as a data source.
  • Customizing the mobile reports.
  • Leverage Dataflows to reduce refresh time of reports

Vous souhaitez devenir un partenaire Power BI ?

Les partenaires certifiés constituent une partie importante de l’équipe ! Trouvez de nouvelles opportunités commerciales, établissez de nouvelles relations, et partagez les talents et l’expérience de votre entreprise avec des utilisateurs Power BI dans le monde entier.

S’inscrire

Découvrez les raisons pour devenir partenaire