Also parse text and HTML OpenSSL feeds

[pombredanne]

OpenSSL publishes an XML feed that we parse but this is not updated
https://www.openssl.org/news/vulnerabilities.html
and https://www.openssl.org/news/secadv/ should be alternative good sources.

[keshav-space]

OpenSSL also provides an updated JSON feed https://www.openssl.org/news/secjson

[pombredanne]

• Structured data are gone: XML data is gone, secadv/ index is gone and secjson/ index is gone.

• The security data comes from a private repository per https://github.com/openssl/web/blob/d3ec13fa40e74fad789462dee4323faa2c6d2991/Makefile#L526 but seems no longer published.

• OpenSSL also has now "premium releases" that are built as explained there https://github.com/openssl/tools/blob/291a580209449f6aebf34e433f3b5f9afbd44c2b/HOWTO-publish-a-release.md

I filed openssl/web#483 upstream as this is problematic.
At the moment the ways to solve this issue would be:

1. Scrape the HTML page to get the list of known vulnerabilities OpenSSL ids and CVEs
2. Fetch the corresponding JSON for each CVE as is https://www.openssl.org/news/secjson/CVE-2002-0659.json
3. Alternatively parse the unstructured text as in https://www.openssl.org/news/secadv/20240115.txt

(note that the date used by OpenSSL as a vulnerability ID is different from the CVE id)

[pombredanne]

Just some status:

• OpenSSL website is reorged and this is a bit of a mess at the new https://openssl-library.org/ because of missing redirects

• https://openssl.org/news/secjson/CVE-2002-0659.json does not redirect at https://openssl-library.org/news/secjson/CVE-2002-0659.json ... there is no directory listing

• The text advisories cannot be listed anymore (no directory listing) but redirect Links to OpenSSL security advisories are broken openssl/web#493

• See other woes at Structured vulnerabilities data are no longer available? openssl/web#483 (comment)

• https://www.cve.org/CVERecord?id=CVE-2024-5535 and https://cveawg.mitre.org/api/cve/CVE-2024-5535 (JSON) seem to be mostly but not exactly the same as the HTML advisories at https://openssl-library.org/news/vulnerabilities/index.html

• JSON advisories seem gone from the new web site

With all this, scraping the web page is likely the way out:

• https://openssl-library.org/news/fips-cve/index.html
• https://openssl-library.org/news/vulnerabilities/index.html

[keshav-space]

Closing this. We now have a fresh v2 importer for OpenSSL to import secjson advisories from https://github.com/openssl/release-metadata/tree/main/secjson.
See #2119