Comparing v6 and v5

[almaz045]

Hi! We have been used v5 version, but decided to test v6. I tried scan different projects and saw interesting things.

1. GO project. For example, I scanned "trivy" repository by v5 and v6. First of all I saw that v6 found more feeds than v5. But I checked the precision of that and saw some of purls in SBOM doesn't really exists in repository. Maybe I am wrong and don't know all nuances but what I see for example:
   pkg:golang/github.com/BurntSushi/toml@v1.2.1 this package I saw in SBOM of v6 depscan, but it absent in v5 SBOM. Then I saw this key in object of this bom's purl:

"properties": [
        {
          "name": "SrcFile",
          "value": "pkg/fanal/analyzer/language/golang/mod/testdata/pkg/mod/github.com/aquasecurity/go-dep-parser@v0.0.0-20230219131432-590b1dfb6edd/go.mod"
        },
        {
          "name": "SrcFile",
          "value": "pkg/fanal/analyzer/language/golang/mod/testdata/pkg/mod/github.com/aquasecurity/go-dep-parser@v0.0.0-20220406074731-71021a481237/go.mod"
        }
      ]

And go to these files to check the presense of this package there but I saw only this:

module github.com/aquasecurity/go-dep-parser

go 1.18

I checked go.mod and go.sum files and didn't see this package.
Can we consider this finding as FP ?

2. Python project. The second interesting thing about scanning the Ansible project is that version 6 also found more findings, but the reason (I think) in that it found findings even for package versions are absent in the SBOM file. These findings are absent in version 5. My guess is that it now finds findings even for packages for which versions aren't specified. However, if a version isn't specified, it's usually assumed that the package is the latest version, whereas here we have findings even for very old package versions (which aren't actually included in the project and SBOM file).

3. Java project. I scanned opensource vulnerable app "vulnado" and v5 found more findings than v6 version. All findings of v5 is correct.

4. JS project. I scanned "cypress" repository and v5 found 50 more findings than v6 (the purls of these findings are presense in both SBOMs files (v5 and v6)).

5. I noticed that some information like licenses, properties of object in components, occurrences in evidence of components are removed from VDR report and presense in SBOM only. Is this a conscious decision?

If you need additional info or checks or artifacts from scans, I can attach

[prabhu]

I think 1 and 2 are related to differences in generated SBOMs. Can you confirm?
3 must be due to the fuzzy search which is an opt-in in v6. Pass --fuzzy-search to enable this.
4 must be due to the npm remote audit which is an opt-in in v6. Pass --risk-audit to enable this.
5 - Correct. These information are usually not needed in a VDR report. This is the place that does the trimming.

[almaz045]

I think 1 and 2 are related to differences in generated SBOMs. Can you confirm?

I rescanned on the latest versions of trivy and ansible repos (previous scans was on old versions of these repos) and results next:

1. For Trivy repo v5 found 39 feeds at all, v6 found 36 feeds at all. v5 found 9 uniq feeds that absent in v6 and v6 found 6 uniq feeds that absent in v5. The count of packages in both boms files is the same (526 purls). The version of cdxgen is 11.11.0. (without --fuzzy-search)
2. For ansible repo v5 no findings, v6 found 26 feeds, but all of these findings don't have purls versions. The count of purls in sbom is same (69, checked for cdxgen 11.11.0 and 12.0.0) (without --fuzzy-search)

[almaz045]

3 must be due to the fuzzy search which is an opt-in in v6. Pass --fuzzy-search to enable this.

This try was with --fuzzy-search opt and next results I got:

SBOM_v5 and SBOM_v6 have the same count of purls (62)

Uniq findings in v5 - 132 (all findings 136)
Uniq findings in v6 - 126 (all findings 143)

The duplicates of CVE ids exist because 1 CVE can refer to several purls (refs).

Because I launched with --fuzzy-search the v6 depscan, it found some FPs.

v5 found 11 uniq feeds (2 of them are FPs) that absent in v6 report.
v6 found 5 uniq feeds (all TPs) that absent in v5 report.

Even with --fuzzy-search the v5 demonstrates more findings in that project.

[almaz045]

4 must be due to the npm remote audit which is an opt-in in v6. Pass --risk-audit to enable this.

I tried with next command:
depscan --profile research -t js -i . -o report --debug --bom-engine CdxgenGenerator --risk-audit and there is no difference with this flag and without it

v5 found 30 uniq feeds (at all 135), v6 (simple and risk scans) found 2 uniq (at all 107). The time spent to scan for v5 - 8 minutes, for v6 - 30 minutes and for v6-risk-audit - 1 hour 24 minutes.

By the way, what the best options to scan projects with maximum useful information?
For example, I should use profile research, it is must have? Or there some other better options?

What difference in lifecycle modes between pre-build, build and post-build?
What difference between technique? Should I pass all variants or I can do not pass it and it will be default ?
When I should use VDRAnalyzer or LifecycleAnalyzer?
Or what better for precision - FrameworkReachability or SemanticReachability ?
And how much will the transition to v6 improve scanning in your opinion?

I would be glad if you could explain the difference to me.

[almaz045]

5 - Correct. These information are usually not needed in a VDR report. This is the place that does the trimming.

By the way, keys "occurrences" and "reachables" is also being cut out. Is this also according to the plan?

[almaz045]

When I scanned with semantic analysis I noticed that there starts 2 scans (maybe 3, but my scans included only pre-build and build phases) and generates 2 sboms. These sboms are almost identical except for differences in some identifiers in the evidense paths. And both sboms may have "occurrences". I noticed these "occurrences" are summed up in VDR report (depscan-universal.vdr.json). For example, if we have 1 occurrence in SBOM pre-build for package A and 1 occurrence in SBOM build for package A then in VDR report we can see for package A info that it has 2 occurrences.

Pre-build phase SBOM:

"occurrences": [
          {
            "location": "src/main/java/com/scalesec/vulnado/VulnadoApplication.java#10"
          },
          {
            "location": "src/main/java/com/scalesec/vulnado/VulnadoApplication.java#7"
          }
        ]
      },

Build phase SBOM:

"occurrences": [
          {
            "location": "src/main/java/com/scalesec/vulnado/VulnadoApplication.java#10"
          },
          {
            "location": "src/main/java/com/scalesec/vulnado/VulnadoApplication.java#7"
          }
        ]
      },

depscan-universal.vdr.json:

"properties":
            [
                {
                    "name": "depscan:prioritized",
                    "value": "true"
                },
                {
                    "name": "depscan:insights",
                    "value": "Endpoint-Reachable\\nUsed in 4 locations"
                }
            ]
        },

These numbers were summed up, although in fact these are 2 unique paths

[prabhu]

Quite a few interesting reports in this thread. Will you be open to contributing some PRs? For reasons unpredictable, I have lost all contributors (and funding) and am finding myself working solo on all projects.