Crafting Data - Augmenting results from Azure DevOps build/test pipelines

To understand the health of the code in QUBO alogrithm, we needed to understand the performance of the algorithm. We ran a sanity test on each checkin and then a more comprehensive test suite each night there was a checkin. The reporting provided Azure DevOps Analytics for build / tests provided was not enough to evalute the alogrithm performance, so we augmented the information from the tests and created this report. information from a test runner point of view.

パートナーに問い合わせる ビデオを見る
Why did we create this?

To understand the health of the code in QUBO algorithm, we needed to understand the
performance of the algorithm. We ran a sanity test on each checkin and then a more
comprehensive test suite each night there was a checkin. The reporting provided
Azure DevOps Analytics for build / tests provided was not enough to evaluate the
algorithm performance, so we augmented the information from the tests and created
this report.

  • What questions are answered?
  • What is the health of the tests for each pipeline ?
  • For each test, how long did the algorithm take to run ?
  • What is the trend for each test run ?
  • Did the algorithm find the correct answer for this test ?
  • If a test failed, was it because of a incorrect result or from maximum time allowed for the test ?

How was this implemented?

Each test is an execution of an algorithm with a given testset. The custom data captured is
the input to the algorithm, amount of time the algorithm is allowed to run, actual run time
for the algorithm, and the result provided by the algorithm.

This custom data is written an Azure Blob in .csv format. An Azure Pipeline then pushes the
test run information (.trx) into to an Azure blob. The files in the Azure Blob are organized
by the build id and the name of the pipeline. 

Finally, the .pbix on each refresh reads the files in the Azure Blob into a semantic model
and the updates the reports and dashboards.

What next?
  • What is the environment for each test (cpu, memory, gpu characteristics) ?
  • Integrate Azure Devops as a data source.
  • Customizing the mobile reports.
  • Leverage Dataflows to reduce refresh time of reports

Power BI パートナーになりたいですか?

認定パートナーは、チームにとって重要な存在です。新しいビジネス チャンスを見つけ、つながりを作り、御社の才能と経験を世界中の Power BI ユーザーと共有してください。

サインアップ

パートナーになるべき理由を確認します