Crafting Data - Augmenting results from Azure DevOps build/test pipelines

To understand the health of the code in QUBO alogrithm, we needed to understand the performance of the algorithm. We ran a sanity test on each checkin and then a more comprehensive test suite each night there was a checkin. The reporting provided Azure DevOps Analytics for build / tests provided was not enough to evalute the alogrithm performance, so we augmented the information from the tests and created this report. information from a test runner point of view.

联系合作伙伴 观看视频
Why did we create this?

To understand the health of the code in QUBO algorithm, we needed to understand the
performance of the algorithm. We ran a sanity test on each checkin and then a more
comprehensive test suite each night there was a checkin. The reporting provided
Azure DevOps Analytics for build / tests provided was not enough to evaluate the
algorithm performance, so we augmented the information from the tests and created
this report.

  • What questions are answered?
  • What is the health of the tests for each pipeline ?
  • For each test, how long did the algorithm take to run ?
  • What is the trend for each test run ?
  • Did the algorithm find the correct answer for this test ?
  • If a test failed, was it because of a incorrect result or from maximum time allowed for the test ?

How was this implemented?

Each test is an execution of an algorithm with a given testset. The custom data captured is
the input to the algorithm, amount of time the algorithm is allowed to run, actual run time
for the algorithm, and the result provided by the algorithm.

This custom data is written an Azure Blob in .csv format. An Azure Pipeline then pushes the
test run information (.trx) into to an Azure blob. The files in the Azure Blob are organized
by the build id and the name of the pipeline. 

Finally, the .pbix on each refresh reads the files in the Azure Blob into a semantic model
and the updates the reports and dashboards.

What next?
  • What is the environment for each test (cpu, memory, gpu characteristics) ?
  • Integrate Azure Devops as a data source.
  • Customizing the mobile reports.
  • Leverage Dataflows to reduce refresh time of reports

想要成为 Power BI 合作伙伴?

认证合作伙伴是团队的重要组成部分!挖据新的商机、建立人脉并与世界各地的 Power BI 用户分享公司的人才和经验。

注册

了解为何要成为合作伙伴