Network Assessment Tool Plus

TechNet gallery link & download location – https://gallery.technet.microsoft.com/lync/Network-Assessment-Tool-33485351

Version history: first release 06/09/2017

Post series: Overview | Running the tool | Processing the data

---

TLDR; Grab the bundle, run the script import to PowerBI, tell me it’s amazing 😛

Summary

Microsoft, back in 2016, released the Microsoft Network Assessment Tool for doing a basic check of connection quality to Skype for Business Online / Teams / O365 (see Randy’s blog for more info). This tool gives you a Pass/Fail based on four network metrics (Packet Loss, Round Trip Time, Jitter, Packet Reorder) for the 90th percentile average of the time you ran the tool.

Whilst a Pass/Fail is helpful it doesn’t “tell you the story” or give you a clue what might be the problem so I created some scripts to loop the tool over a long period of time and a PowerBI file that can help visualise the data to dig into where there might be issues.

This post runs through how to run the Microsoft tool continuously and then import the data into PowerBI so you get a lovely looking dashboard like this:

Network Assessment Tool Plus – package contents

Currently the package contains

• MS Assessment looping script (PowerShell)
  • Network_Assessment_Plus_Looping_Script_V3.txt
• PowerBI file for analysing results
  • Network_Assessment_Plus_PowerBI_v2.pbix
• CSV file to update with your results
  • Network_Assessment_Combined.csv
• Sample data
  • NetworkTest_Sample_Results.tsv
  • NetworkTest_Sample_Results.tsv.csv
• Documentation
  • Running the Network Assessment Tool Plus.pdf

MS Tool Summary

Before we dive in it’s important to recap on a few a few basic things. The Microsoft tool sends 17 seconds worth of audio to your closest Microsoft “Edge” where it is returned and the tool gathers the four metrics mentioned above about the transaction. These results are written out to a .tsv file & while you can run the test more than once you need to be careful as re-running the tool overwrites all previous results (hence the script we are going to use in a minute)!

What the tool does not do:

• It is not designed to perform stress or load testing
• It does not show you data trends
• It does not represent “real world” conditions as the audio sample size is very small and QoS is unlikely to be applied to the traffic

I like to use this tool to give a rough indication about the underlying connection quality, both LAN/WAN/WiFi and Internet connections from multiple locations where your users might connect from. We can then take this data and treat it like the “canary in the mine” to get a feel for where we might have problems at deployment time. If you want a better test run under more true-to-life conditions then there are other products and services that you can purchase (contact me if you want to discuss further).

With those caveats in place lets get cracking my intrepid network explorer

Choosing your test locations

The first thing we need to do is identify where on your network you should run the tool. My suggestion is in as many places as possible as the more data you get the better your insight will be. Remember that the tool by default sends data to the nearest “Microsoft Edge” so if you run this from another part of the world you aren’t testing connectivity to your tenant location but merely to the closest MS datacentre (this would however mirror your behaviour if you had RHM enabled).

In the sample diagram we have:

• Site1 – Local internet breakout
• Site2 – Connecting to internet via WAN then Datacentre breakout
• Site 3 – Datacentre with internet breakout – the difference between Site 2 and Site 3 will give you the WAN performance
• Site 4 – Site in a different geographical region

Gathering data

Detailed instructions for running the tool looping script can be found in this seperate post (easier to update as I improve things).

Once you have picked your locations find a desktop/laptop or server running Windows 7 or above then install the x86 Visual C++ run-time components for Visual Studio 2015 (Make sure you ALWAYS install the x86 version regardless of your OS “bitness”). At the moment my script doesn’t do this pre-req check but it will download & unzip the tool from the MS website for you.

Grab a copy of the bundle from TechNet Gallery, extract it to a local directory, find the looping tool in the root and rename it from a .txt to a .ps1 PowerShell file. If you want you can just jump straight in and fire up the script from a PowerShell window (does not need admin rights).

What the script file will do:

• Check if the MS Tool exists in the current directory
  • If not download & unzip the tool from Microsoft
• Write out some extra info to a separate CSV file if not already there named for the current PC
  • Current PC culture
  • Example date / time
  • Current connection type (WiFi/wired)
  • Current IP address
• Run the tool & check for a results file
• Grab the results file & check for malformed results
• Append the results into a combined .TSV file named for the current PC
• Re-run the tool
• Repeat

Credit for the first version of this looping tool goes to Tom Arbuthnot, subsequent versions and bugs are my fault

Note: Also included in the download ZIP is a more end-user focused version of the guide if you wanted to send this out to home workers for example.

If everything is working you should see two new files being created in the script’s directory, a .csv and a .tsv. This is what we need to work with for the PowerBI magic…

Processing the results

Next up we want to pull those results into something useful for processing in PowerBI. We need to take the .tsv files from the test nodes, add a site code, convert them into .csv files and then combine them for import into PowerBI.

Full info on prepping & processing the files are in this seperate post.

Summary steps:

• Import into Excel
• Add a “Site” column and fill in
• Convert the format for the date/time column
• Save out as a .csv
• Combine all .csv files together

Analysing the results

With all the data imported you should now see something similar to the picture at the top of this post and can start clicking about to dig into how your network is performing.

An important thing to remember about PowerBI is that it’s great for making things interactive. As you click about the screen and select different elements the rest of the data on display will adjust dynamically to reflect your selections.

I’ll be writing a more detailed post showing how the PowerBI file is constructed & how to use it but here are the highlights to get you started:

Site slice

Use the slicer to select which site you want to view data for, use ctrl+click to select more than one

Call stats

Shows a total of all the calls in the current view

• Total
• Total calls classed as good
• Total calls with a “warning” flag
• Total calls with a “bad” flag
• Calls classed as invalid

Note: calls can be both “warning” and “bad” for different metrics

90th percentile average

Shows the average of the “90th percentile” data for each metric in the current display. Microsoft recommend only using the 90th percentile of the data (excluding the best & worst results). If you exclude the invalid calls these values should match up with the output from the official tool results analyser.

Average of everything

Shows the average of all the results in the current display

Call slices

• Invalid calls
  • 0 – Remove calls classed as “invalid” from the display
  • 1 – Only show “invalid” calls
• Warn
  • 0 – Show calls with no “warning” flag set in any metric
  • 1 – Show calls with a “warning” flag set in any metric
• Bad
  • 0 – Show calls with no “bad” flag set in any metric
  • 1 – Show calls with a “bad” flag set in any metric

Note: the same call can be flagged as both “bad” and “warning” for different fields

Working hour slices

• Weekday
  • 0 – Only show calls Monday-Friday
  • 1 – Only show calls Saturday/Sunday
• Working Hours
  • 0 – Only show calls from between 20.00 and 8.00
  • 1 – Only show calls between 8.00 and 20.00

Date picker

Adjust the slider to change the date range for the view

Time of day stacker

Stack graph showing total calls in each time period (total height), the shading indicates the number of good, warning & bad calls. This should give you a good overview of any issues during working hours – i.e. 9AM when users load up the network, or 22.00 when backups kick in.

The hours can be clicked to filter the rest of the data views, hold CTRL to select more than one

Ideally you want all of these bars to be green with no yellow or red & all the same height!

Metric slices

Each metric has a summary table next to it showing “good”, “warning” and “bad” count as well as two slices to filter the data just on that metric

Metrics data

The main body of the data report showing a visualisation of the average for each metric over time. Each graph has a red & yellow dashed line showing the warning & critical levels

Next steps

Once you have surveyed your network and run your tests you should have a pretty good idea where you might need to do a bit of further investigation..

It might be that you need to do a bit of further investigation into certain network segments or locations. If this is the case there are other non-free products and services which can help (such as Modality Systems – Impact assessment / Diagnostics platform)

It is important to remember what we talked about at the start, this MS tool is NOT designed to simulate true real-world scenarios, the call volumes are too low & the call duration is to short. What it will do is give you an indication of the sutiability of your underlying network infrastructure!

Network Assessment Tool Plus Wish List

• Script to automate processing & combining the results.tsv files
• Assessment tool pre-req check, download & install
• Better compatibility for older PowerShell editions

Lastly

Lastly, thank you for reading this post and hopefully having a play with my tool & PowerBI report, as you can imagine this has taken a lot of time and effort to pull together and it would be great if you can take a couple of minutes to drop me a comment here or on TechNet. Nothing huge, just something about how you used it, or any suggestions for updates or improvements welcome too!

Thanks

Ben.

License

The script is available under the OpenSource MIT license – https://opensource.org/licenses/MIT

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.