Only one Too Long; Did Read post this week, The State of XIoT report for 1H 2022 by Claroty’s Team82. The content and analysis of this report was uneven. There is a lot to commend the team for, but also significant sections that left me shaking my head. This is a reimagining of previous reports, so it’s hopeful that future Claroty reports will build on the better aspects of this one.

Regwall: Yes, https://claroty.com/resources/reports/state-of-xiot-security-1h-2022

Target Audience: IoT (or XIoT) experts

Length & Read time: 35 pages, 30-60 minutes, longer if you are not an IoT expert. I finished reading the report in 75 minutes but required extra time to review some of the terminology and the Purdue Model.

Grade: B. Despite some parts being hard to digest, this has more going for it than most.

Overall Impression: I dislike the creation of new initialisms/acronyms in reports, but I think Claroty can get away with it this time. ‘XIoT’ stands for the Extended Internet of Things, meaning medical devices, video cameras, embedded devices, and a whole host of other general connected ‘things’. However, the report often uses the initialism for many of these things without clarifying what they mean and how they are used in the report. Lack of definition is a reoccuring theme of the report, from defining terms, to explaining the statistics used, to plots with no titles or captions.

Be prepared to spend some time identifying and understanding the most important parts of this report on your own. Most of the text is a reading of the visualizations, with confusing context and analysis. It may be because IoT/XIoT isn’t my main area of interest, but I think it’s because I don’t like having plots read at me. The writers left too much to the reader to figure out.

Despite the uneven delivery of the report, I still suggest reading it if you’re interested in IoT in its myriad forms. Several sections contain Key Events and are worth reading on their own. More than anything else, it’s the Mitigations/Remediations section I would point readers at, starting on page 22. Not only does Team82 give specific suggestions, they provide data to show why specific recommendations should be the reader’s priority. This section is why think the report is above average, but in need of tender loving care and focus to make it truly shine.

The Good: Let’s start with the low hanging fruit. The Team82 report clearly credits the contributions of the researchers, writers, and data scientists who worked on the effort, which is not as common as it should be. The visualizations are generally clear, though there’s more than a few to cover in the next section of the review. The report sticks with simple visualizations, mainly donut and bar charts. It is a visually pleasing report in most ways. Its layout almost makes some sections into ready to use slides. The colors chosen for the plots make them accessible and are color blind friendly, though the variation of colors in the donut charts are confusing.

As mentioned before, the Mitigation/Remediations section and four Key Events segments are what makes the report above average. We’re past a time when the security events that could happen are just something we theorize about. From the Russia-Ukraine war to ransomware attacks, we’re seeing attacks on a daily basis that affect real businesses and real people.

I like the Team82 section of the report; it highlights the group’s technical knowledge without diving into a marketing spin. A report should show off the author’s experise, but not blatantly scream ‘Buy our product!’. That said, the section belongs further down in the report. Showing off the data and findings should always come first.

While the report is 35 pages in length, at least 8 pages are separators and the covers, which means it’s really closer to 25 pages of content. Even at that, it’s not a short report and there’s plenty of sections that need extra time to digest. The text isn’t too dense, but it requires some concentration to read.

The Bad: The report is very uneven in its delivery of intelligence and how the various sections relate to each other. The data is represented unclearly in many cases, with loosely related data being represented together without an attempt at defining the different threads. In my view, multiple sections are mislabeled and don’t fit the function they advertise.

The Executive Summary isn’t a summary at all, it’s an opening statement that fails at telling the reader what’s most important in the report. Too much time is spent on the opening arguments, without really telling the reader what they’ll get out of the report. It does tell us who should read it, but not why. I’m very keen on telling the reader from the first page what I want them to take from a report and highlighting the primary talking points I want them to know. The Executive Summary is more akin to a first research section than it is a summary.

One of the initialisms from the report, IoMT, is first used in the donut chart on page 5, but the term isn’t defined until page 15. And I still don’t know what the author’s definition of ‘IT’ is. Even if this is meant for experts in IoT, any initialism or acronym should be spelled out the first time it’s used. I might even argue that ‘XIoT’ should have been defined on the cover.

The figures aren’t numbered, with many lacking a title or caption. The donut chart on page 6 lacks both caption and title. While you can tease out which of the talking points to the left it refers to, you shouldn’t have to. The same page is a good example of missing analysis in many parts of the report. Four different boxes of text are displayed with no clear relationship and a plot lacking context to give it meaning. This is all still in the Executive Summary.

Readers should be clearly shown how the data, the plots, and the analysis are intertwined in a good report. In this case the reader is left to their own devices far too often. Page 26 typifies my concern, with a series of percentages that are dropped onto the page with almost no supporting text.

While the Mitigation/Remediations section was very good, the Recommendations section simply presents the same talking points without the benefit of the data. I think the two sections should have been combined into one, with a single page summary of the report at the end.

Overall, the Team82 report on XIoT is an instrument built of strong materials, but in need of polishing and another generation of design work. It has several very strong sections that may never get read because it loses the reader almost immediately. Kind of like these TL;DR posts.