The Thursday edition of ‘Too Long; Did Read’ focuses on Fortinet’s Global Threat Landscape Report for the first half of 2022. In my experience, 7-8 weeks to go from data collection to a published report is a relatively tight deadline, but it means this data is still timely. I applaud Fortinet for making this report available without having to register, which is a rarity. The failure of this report is the use of the term, “prevalence”. I suspect most readers have never seen this term used in its data science context and no effort is made to clarify what it means. My suggestion is to read the text, but generally ignore the plots and graphs.
RegWall: No! https://www.fortinet.com/content/dam/fortinet/assets/threat-reports/threat-report-1h-2022.pdf
Target Audience: Technical, primarily for readers in the IPS space.
Length & Read time: 17 pages, 30-45 minutes. Taking notes, I spent 75 minutes with this report, with additional time researching prevalence in statistics.
Grade: C, average for the security industry.
Overall Impression: I was excited to read this report when I first saw it, but found myself becoming more confused as I read through the text and tried to make sense of the data visualizations. The Overview and Key Highlights section sums up the main talking points, but most lack the specificity I’d like to see.
If you only have a few minutes to devote to the report, read these sections: ‘Vulnerabilities in OT’, pages 7 & 8; ‘Ransomware Roundup’, pages 12-14. I also liked the review of wipers targeting the Ukraine and spilling over into other countries, pages 14-16. But I can’t recommend the report due to difficult to understand data representation choices.
The Good: As mentioned above, there are a few sections that bouy this report. Figure 5 uses the idea of prevalence in a way that’s understandable, unlike how it’s expressed elsewhere in the report. The TTPs table uses color well, highlighting how often different types of techniques are used. I like figure 12, a map of the countries affected by the wipers tied to the Russia-Ukraine war. As a product of the US education system, I’m aware than many of us have a hard time remembering where European countries are in realationship to each other. I’ve been to all the countries highlighted and still need to look at the map.
The report has a large number of links leading to other resources. Half lead to other Fortinet reports or posts, half take the reader to organizations like MITRE and FIRST. I’m generally happy to see readers given an opportunity to learn more. It’s clear a lot of knowledge and experience has gone into the report.
The Bad: I am not a data scientist and I’m the first to admit there are significant holes in my education on the topic. But I believe I have significantly more experience with data science than then majority of readers targeted by the Global Threat Landscape Report. Which is why when I started seeing the term ‘prevalence’ used with Figure 2 and beyond, I was confused. I’ve never seen it used in another report in the security industry. I didn’t know what it meant in context, I didn’t understand the significance of the percentage, and I still don’t understand the specifics of how it was in various visualizations.
In statistics, prevalence is the proportion of individuals in a population who have a specific characteristic at a certain time period.
Statology, https://www.statology.org/prevalence-in-statistics/
I understand the standard English definition of ‘prevalence’ as ‘commonness’. But used as a percentage measurement for exploits, vulnerabilities, etc. it’s hard to make sense of. When it’s used in Figure 5 and shows ‘1 in 10K’ on the axis, it makes sense, but every other use of ‘prevalence’ is confusing. I like drawing my own conclusions and over half the plots are useless for generating your own analysis. I found reading these visualizations a very frustrating effort.
The paper lacks credits and a methodology section, both of which I expect to see in a report. A few paragraphs spent explaining how the data was collected, how ‘prevalence’ was calculated, and what the percentages mean would have made a large part of my complaints moot. Even with an explination, I don’t think prevalence is the best way to represent the data used, but I’m open to hearing more.
Rather than having a conclusion, this report titles their closing section as, ‘Ending on a High Note’. A more accurate description would be, ‘Ending with a Marketing Call to Action’. It doesn’t summarize any of the intelligence in the report, there are no conclusions, it’s a CTA section, pure and simple. Marketing is a valuable part of any enterprise, but a report is better served highlighting the intelligence and experience of the team. If a reader finds value in your intelligence, they’ll look at your product without being hit on the head with product pitches.
Please, please, please, honor the work of the authors by crediting them publicly in future reports.