{"id":41020,"date":"2021-09-17T09:21:14","date_gmt":"2021-09-17T07:21:14","guid":{"rendered":"https:\/\/www.pentestfactory.de\/cve-quick-search-implementing-our-own-vulnerability-database\/"},"modified":"2024-07-26T10:30:18","modified_gmt":"2024-07-26T08:30:18","slug":"cve-quick-search-implementing-our-own-vulnerability-database","status":"publish","type":"post","link":"https:\/\/www.pentestfactory.de\/en\/cve-quick-search-implementing-our-own-vulnerability-database\/","title":{"rendered":"CVE Quick Search: Implementing our own vulnerability database"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-post\" data-elementor-id=\"41020\" class=\"elementor elementor-41020 elementor-40895\" data-elementor-post-type=\"post\">\n\t\t\t\t\t\t<section class=\"has_eae_slider elementor-section elementor-top-section elementor-element elementor-element-fb49da3 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-eae-slider=\"14837\" data-id=\"fb49da3\" data-element_type=\"section\" data-e-type=\"section\" data-settings=\"{&quot;ekit_has_onepagescroll_dot&quot;:&quot;yes&quot;}\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"has_eae_slider elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-9d6fe02\" data-eae-slider=\"81896\" data-id=\"9d6fe02\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-ae779c9 elementor-widget elementor-widget-text-editor\" data-id=\"ae779c9\" data-element_type=\"widget\" data-e-type=\"widget\" data-settings=\"{&quot;ekit_we_effect_on&quot;:&quot;none&quot;}\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Not only for penetration testing it is interesting to know, which vulnerabilities exist for a certain software product. Also from the perspective of an IT team it can be useful to quickly obtain information about an employed product version. So far various databases existed for these queries like e.g., <a href=\"https:\/\/nvd.nist.gov\/vuln\/search\" rel=\"nofollow noopener\" target=\"_blank\">https:\/\/nvd.nist.gov\/vuln\/search<\/a>, <a href=\"https:\/\/cvedetails.com\" rel=\"nofollow noopener\" target=\"_blank\">https:\/\/cvedetails.com<\/a> or <a href=\"https:\/\/snyk.io\/vuln\" rel=\"nofollow noopener\" target=\"_blank\">https:\/\/snyk.io\/vuln<\/a><\/p>\n<p>However, during the last years, we could identify several issues with these databases:<\/p>\n<ul type=\"disc\">\n<li>Many databases only index vulnerabilities for certain product groups (e.g., Snyk: Web Technologies)<\/li>\n<li>Many databases search for keywords in full-text descriptions. Searching for specific product versions is not precise.<\/li>\n<li>Many databases are outdated or list incorrect information<\/li>\n<\/ul>\n<p style=\"text-align: center;\"><img fetchpriority=\"high\" decoding=\"async\" class=\"aligncenter wp-image-40898 size-full\" src=\"https:\/\/www.pentestfactory.de\/wp-content\/uploads\/1.png\" alt=\"\" width=\"2560\" height=\"715\" title=\"\" srcset=\"https:\/\/www.pentestfactory.de\/wp-content\/uploads\/1.png 2798w, https:\/\/www.pentestfactory.de\/wp-content\/uploads\/1-300x84.png 300w, https:\/\/www.pentestfactory.de\/wp-content\/uploads\/1-1024x286.png 1024w, https:\/\/www.pentestfactory.de\/wp-content\/uploads\/1-768x214.png 768w, https:\/\/www.pentestfactory.de\/wp-content\/uploads\/1-1536x429.png 1536w, https:\/\/www.pentestfactory.de\/wp-content\/uploads\/1-2048x572.png 2048w\" sizes=\"(max-width: 2560px) 100vw, 2560px\" \/><em>Figure: Incorrect vulnerability results for Windows 10<\/em><\/p>\n<p style=\"text-align: center;\"><img decoding=\"async\" class=\"alignnone wp-image-40902 size-full aligncenter\" src=\"https:\/\/www.pentestfactory.de\/wp-content\/uploads\/3.png\" alt=\"\" width=\"1000\" height=\"438\" title=\"\" srcset=\"https:\/\/www.pentestfactory.de\/wp-content\/uploads\/3.png 1000w, https:\/\/www.pentestfactory.de\/wp-content\/uploads\/3-300x131.png 300w, https:\/\/www.pentestfactory.de\/wp-content\/uploads\/3-768x336.png 768w\" sizes=\"(max-width: 1000px) 100vw, 1000px\" \/><em>Figure: Keyword search returns a different product than the originally searched for product<\/em><\/p>\n<p>This is why we decided to implement our own solution. We considered the following key points:<\/p>\n<ul type=\"disc\">\n<li>Products and version numbers can be searched using unique identifiers. This allows a more precise search query.<\/li>\n<li>The system performs a daily import of the lastest vulnerability data from the National Institute of Standards and Technology (NIST). Vulnerabilities are thus kept up to date and have a verified CVE entry.<\/li>\n<li>The system is based on Elastic Stack <a href=\"https:\/\/www.elastic.co\/de\/elastic-stack\/\" rel=\"nofollow noopener\" target=\"_blank\">https:\/\/www.elastic.co\/de\/elastic-stack\/<\/a> to query and visualize data in real time.<\/li>\n<\/ul>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"has_eae_slider elementor-section elementor-top-section elementor-element elementor-element-9cf3c17 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-eae-slider=\"84259\" data-id=\"9cf3c17\" data-element_type=\"section\" data-e-type=\"section\" data-settings=\"{&quot;ekit_has_onepagescroll_dot&quot;:&quot;yes&quot;}\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"has_eae_slider elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-3708231\" data-eae-slider=\"4141\" data-id=\"3708231\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-efa5c5a elementor-widget elementor-widget-text-editor\" data-id=\"efa5c5a\" data-element_type=\"widget\" data-e-type=\"widget\" data-settings=\"{&quot;ekit_we_effect_on&quot;:&quot;none&quot;}\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<h3>Technical Implementation: NIST NVD &amp; Elastic Stack<\/h3>\n<p>Upon finding vulnerabilities in products, security researchers commonly register a CVE entry per vulnerability. These CVE entries are given a unique identifier, detailed vulnerability information, as well as a general description.<\/p>\n<p>They can be registered at <a href=\"https:\/\/cve.mitre.org\" rel=\"nofollow noopener\" target=\"_blank\">https:\/\/cve.mitre.org<\/a> and are indexed in the National Vulnerability Database (NVD) in real time (<a href=\"https:\/\/cve.mitre.org\/about\/cve_and_nvd_relationship.html\" rel=\"nofollow noopener\" target=\"_blank\">https:\/\/cve.mitre.org\/about\/cve_and_nvd_relationship.html<\/a>). NIST publishes these data sets publicly and freely, which contain all registered vulnerabilities. We use this data stream as a basis for our own database.<\/p>\n<p>The technical details of the data import and subsequent provisioning are illustrated as follows:<\/p>\n<p style=\"text-align: center;\"><img decoding=\"async\" class=\"aligncenter wp-image-40904 size-full\" src=\"https:\/\/www.pentestfactory.de\/wp-content\/uploads\/4.png\" alt=\"\" width=\"1198\" height=\"499\" title=\"\" srcset=\"https:\/\/www.pentestfactory.de\/wp-content\/uploads\/4.png 1198w, https:\/\/www.pentestfactory.de\/wp-content\/uploads\/4-300x125.png 300w, https:\/\/www.pentestfactory.de\/wp-content\/uploads\/4-1024x427.png 1024w, https:\/\/www.pentestfactory.de\/wp-content\/uploads\/4-768x320.png 768w\" sizes=\"(max-width: 1198px) 100vw, 1198px\" \/><em>Figure: Overview of the technical components of the vulnerability database<\/em><\/p>\n<h4><strong>1. Daily import of vulnerability data from the NIST NVD<\/strong><\/h4>\n<p>The data sets are organized by year numbers and refreshed daily by NIST. Every night we download the latest files onto our file server.<\/p>\n<h4>2. Pre-Processing of vulnerability data<\/h4>\n<p>Afterwards the files are pre-processed to make them compatible with the Elastic Stack Parser. One process that happens here is the expansion of all JSON files: The downloaded files contain JSON objects, however they are often nested, which makes it harder to identify single objects for the parser. We read the JSON and write all object seperators into separate lines. This way we can use a regex ( &#8216;^{&#8216; ) to precisely determine, when a new object begins.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"has_eae_slider elementor-section elementor-top-section elementor-element elementor-element-64f6a33 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-eae-slider=\"99611\" data-id=\"64f6a33\" data-element_type=\"section\" data-e-type=\"section\" data-settings=\"{&quot;ekit_has_onepagescroll_dot&quot;:&quot;yes&quot;}\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"has_eae_slider elementor-column elementor-col-20 elementor-top-column elementor-element elementor-element-4f9aa5b\" data-eae-slider=\"32679\" data-id=\"4f9aa5b\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap\">\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t<div class=\"has_eae_slider elementor-column elementor-col-20 elementor-top-column elementor-element elementor-element-a39aa87\" data-eae-slider=\"34199\" data-id=\"a39aa87\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-8aa68d9 elementor-widget elementor-widget-image\" data-id=\"8aa68d9\" data-element_type=\"widget\" data-e-type=\"widget\" data-settings=\"{&quot;ekit_we_effect_on&quot;:&quot;none&quot;}\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"342\" height=\"344\" src=\"https:\/\/www.pentestfactory.de\/wp-content\/uploads\/5.png\" class=\"attachment-large size-large wp-image-40907\" alt=\"\" srcset=\"https:\/\/www.pentestfactory.de\/wp-content\/uploads\/5.png 342w, https:\/\/www.pentestfactory.de\/wp-content\/uploads\/5-298x300.png 298w, https:\/\/www.pentestfactory.de\/wp-content\/uploads\/5-150x150.png 150w\" sizes=\"(max-width: 342px) 100vw, 342px\" title=\"\">\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t<div class=\"has_eae_slider elementor-column elementor-col-20 elementor-top-column elementor-element elementor-element-f23a1b2\" data-eae-slider=\"75300\" data-id=\"f23a1b2\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-4c85da0 elementor-widget elementor-widget-spacer\" data-id=\"4c85da0\" data-element_type=\"widget\" data-e-type=\"widget\" data-settings=\"{&quot;ekit_we_effect_on&quot;:&quot;none&quot;}\" data-widget_type=\"spacer.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"elementor-spacer\">\n\t\t\t<div class=\"elementor-spacer-inner\"><\/div>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-eede9e0 elementor-view-default elementor-widget elementor-widget-icon\" data-id=\"eede9e0\" data-element_type=\"widget\" data-e-type=\"widget\" data-settings=\"{&quot;ekit_we_effect_on&quot;:&quot;none&quot;}\" data-widget_type=\"icon.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"elementor-icon-wrapper\">\n\t\t\t<div class=\"elementor-icon\">\n\t\t\t<i aria-hidden=\"true\" class=\"icon icon-arrow-right\"><\/i>\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t<div class=\"has_eae_slider elementor-column elementor-col-20 elementor-top-column elementor-element elementor-element-59a5c9b\" data-eae-slider=\"68786\" data-id=\"59a5c9b\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-fb0c259 elementor-widget elementor-widget-image\" data-id=\"fb0c259\" data-element_type=\"widget\" data-e-type=\"widget\" data-settings=\"{&quot;ekit_we_effect_on&quot;:&quot;none&quot;}\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"450\" height=\"467\" src=\"https:\/\/www.pentestfactory.de\/wp-content\/uploads\/6.png\" class=\"attachment-large size-large wp-image-40909\" alt=\"\" srcset=\"https:\/\/www.pentestfactory.de\/wp-content\/uploads\/6.png 450w, https:\/\/www.pentestfactory.de\/wp-content\/uploads\/6-289x300.png 289w\" sizes=\"(max-width: 450px) 100vw, 450px\" title=\"\">\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t<div class=\"has_eae_slider elementor-column elementor-col-20 elementor-top-column elementor-element elementor-element-e43b8d1\" data-eae-slider=\"29505\" data-id=\"e43b8d1\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap\">\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"has_eae_slider elementor-section elementor-top-section elementor-element elementor-element-8f16a41 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-eae-slider=\"99870\" data-id=\"8f16a41\" data-element_type=\"section\" data-e-type=\"section\" data-settings=\"{&quot;ekit_has_onepagescroll_dot&quot;:&quot;yes&quot;}\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"has_eae_slider elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-bd6f478\" data-eae-slider=\"89650\" data-id=\"bd6f478\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-ecc1461 elementor-widget elementor-widget-text-editor\" data-id=\"ecc1461\" data-element_type=\"widget\" data-e-type=\"widget\" data-settings=\"{&quot;ekit_we_effect_on&quot;:&quot;none&quot;}\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Furthermore we strip the file of all unneeded metadata (e.g., autor, version information, etc.), which leaves only the CVE entries in the file as sequential JSON objects.<\/p>\n<h4>3. Reading in the pre-processed vulnerability data using Logstash<\/h4>\n<p>After the pre-processing, our Logstash parser is able to read the individual lines of the files using the Multiline Codec (<a href=\"https:\/\/www.elastic.co\/guide\/en\/logstash\/current\/plugins-codecs-multiline.html\" rel=\"nofollow noopener\" target=\"_blank\">https:\/\/www.elastic.co\/guide\/en\/logstash\/current\/plugins-codecs-multiline.html<\/a>). Every time a complete JSON object is read in, Logstash forwards this CVE object to our Elasticsearch instance.<\/p>\n<\/p>\n<h3>The CVE Quick Search &#8211; Data formats and vulnerability queries<\/h3>\n<p>After all CVE entries were read and stored in the Elasticsearch database, we have to understand, which format these entries have and how we can search them for specific products and product vulnerabilities. Our final result is illustrated in the following screenshot: Using unique identifiers, we can return exact vulnerability reports for the queried product version.<\/p>\n<p style=\"text-align: center;\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-40932 size-full\" src=\"https:\/\/www.pentestfactory.de\/wp-content\/uploads\/2021-09-17-09_56_10-Clipboard.png\" alt=\"\" width=\"1018\" height=\"403\" title=\"\" srcset=\"https:\/\/www.pentestfactory.de\/wp-content\/uploads\/2021-09-17-09_56_10-Clipboard.png 1018w, https:\/\/www.pentestfactory.de\/wp-content\/uploads\/2021-09-17-09_56_10-Clipboard-300x119.png 300w, https:\/\/www.pentestfactory.de\/wp-content\/uploads\/2021-09-17-09_56_10-Clipboard-768x304.png 768w\" sizes=\"(max-width: 1018px) 100vw, 1018px\" \/><em>Figure: Preview of our vulnerability query frontend<\/em><\/p>\n<h4>1. Format of product versions<\/h4>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"has_eae_slider elementor-section elementor-top-section elementor-element elementor-element-8d6ac6b elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-eae-slider=\"55911\" data-id=\"8d6ac6b\" data-element_type=\"section\" data-e-type=\"section\" data-settings=\"{&quot;ekit_has_onepagescroll_dot&quot;:&quot;yes&quot;}\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"has_eae_slider elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-ba568e7\" data-eae-slider=\"48497\" data-id=\"ba568e7\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-2035eff elementor-widget elementor-widget-text-editor\" data-id=\"2035eff\" data-element_type=\"widget\" data-e-type=\"widget\" data-settings=\"{&quot;ekit_we_effect_on&quot;:&quot;none&quot;}\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>The general format of product versions is specified in the NIST specification. Section 5.3.3 gives a short overview (<a href=\"https:\/\/nvlpubs.nist.gov\/nistpubs\/Legacy\/IR\/nistir7695.pdf\" rel=\"nofollow noopener\" target=\"_blank\">https:\/\/nvlpubs.nist.gov\/nistpubs\/Legacy\/IR\/nistir7695.pdf<\/a>):<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"has_eae_slider elementor-section elementor-top-section elementor-element elementor-element-b8a6d3e elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-eae-slider=\"30394\" data-id=\"b8a6d3e\" data-element_type=\"section\" data-e-type=\"section\" data-settings=\"{&quot;ekit_has_onepagescroll_dot&quot;:&quot;yes&quot;}\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"has_eae_slider elementor-column elementor-col-50 elementor-top-column elementor-element elementor-element-23690d6\" data-eae-slider=\"65247\" data-id=\"23690d6\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-691d535 elementor-widget elementor-widget-image\" data-id=\"691d535\" data-element_type=\"widget\" data-e-type=\"widget\" data-settings=\"{&quot;ekit_we_effect_on&quot;:&quot;none&quot;}\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"602\" height=\"693\" src=\"https:\/\/www.pentestfactory.de\/wp-content\/uploads\/8.png\" class=\"attachment-large size-large wp-image-40913\" alt=\"\" srcset=\"https:\/\/www.pentestfactory.de\/wp-content\/uploads\/8.png 602w, https:\/\/www.pentestfactory.de\/wp-content\/uploads\/8-261x300.png 261w\" sizes=\"(max-width: 602px) 100vw, 602px\" title=\"\">\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t<div class=\"has_eae_slider elementor-column elementor-col-50 elementor-top-column elementor-element elementor-element-7265df2\" data-eae-slider=\"81878\" data-id=\"7265df2\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-c2e38e1 elementor-widget elementor-widget-text-editor\" data-id=\"c2e38e1\" data-element_type=\"widget\" data-e-type=\"widget\" data-settings=\"{&quot;ekit_we_effect_on&quot;:&quot;none&quot;}\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><em>cpe:2.3:part:vendor:product_name:version:update:edition:sw_edition:target_sw:target_hw:language:other<\/em><\/p>\n<ul type=\"circle\">\n<li><strong>part<\/strong>: either &#8216;a&#8217; (application), &#8216;o&#8217; (operating system) or &#8216;h&#8217; (hardware)<\/li>\n<li><strong>vendor<\/strong>: unique identifier of the product vendor<\/li>\n<li><strong>product_name<\/strong>: a unique name identifier of the product<\/li>\n<li><strong>version<\/strong>: the version number of the product<\/li>\n<li><strong>edition<\/strong>: deprecated<\/li>\n<li><strong>sw_edition<\/strong>: Version for identifiying different market versions<\/li>\n<li><strong>target_sw<\/strong>: Software environment the product is used with\/in<\/li>\n<li><strong>target_hw<\/strong>: Hardware environment the product is used with\/in<\/li>\n<li><strong>language<\/strong>: Supported language<\/li>\n<li><strong>other<\/strong>: other annotations<\/li>\n<\/ul>\n<p>A colon is used as a separating character. Asterisk (*) is used as a wildcard symbol.<\/p>\n<p>In our screenshot: &#8220;cpe:2.3:o:juniper:junos:17.4r3:*:*:*:*:*:*:*&#8221; we can determine that the operating system JunOS of the vendor Juniper in version 17.4r3 is prone to a vulnerability.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"has_eae_slider elementor-section elementor-top-section elementor-element elementor-element-95eee1f elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-eae-slider=\"82907\" data-id=\"95eee1f\" data-element_type=\"section\" data-e-type=\"section\" data-settings=\"{&quot;ekit_has_onepagescroll_dot&quot;:&quot;yes&quot;}\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"has_eae_slider elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-78a766f\" data-eae-slider=\"66453\" data-id=\"78a766f\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-7657c80 elementor-widget elementor-widget-text-editor\" data-id=\"7657c80\" data-element_type=\"widget\" data-e-type=\"widget\" data-settings=\"{&quot;ekit_we_effect_on&quot;:&quot;none&quot;}\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Looking at the JSON file, it becomes apparent that there are two formats that are used to store the version number of a vulnerability.<\/p>\n<ul type=\"disc\">\n<li>Format 1: Using the attributes &#8220;versionStartIncluding\/versionStartExcluding&#8221; and &#8220;versionEndIncluding\/versionEndExcluding&#8221; a range of vulnerable versions is defined.<\/li>\n<\/ul>\n<ul type=\"disc\">\n<li>Format 2: A single vulnerable software version is stored in &#8220;cpe23Uri&#8221;.<\/li>\n<\/ul>\n<h4>2. Querying the database<\/h4>\n<p>To query the database for specific products, an easy interface to find correct product identifiers is required. We have decided to implement this component using JavaScript Auto-Complete, that displays products and associated CPE identifiers dynamically:<\/p>\n<p style=\"text-align: center;\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-40914 size-full\" src=\"https:\/\/www.pentestfactory.de\/wp-content\/uploads\/9.png\" alt=\"\" width=\"708\" height=\"272\" title=\"\" srcset=\"https:\/\/www.pentestfactory.de\/wp-content\/uploads\/9.png 708w, https:\/\/www.pentestfactory.de\/wp-content\/uploads\/9-300x115.png 300w\" sizes=\"(max-width: 708px) 100vw, 708px\" \/><em>Figure: Autocomplete mechanism of the query frontend<\/em><\/p>\n<p>After a choice was made, the vulnerabilities matching the specific product identifier can be queried.<\/p>\n<h3> <\/h3>\n<h3>Outlook: Kibana &#8211; Visualising vulnerabilities and trends<\/h3>\n<p>A big advantage that storing vulnerability data in an Elasticsearch database has, is its direct connection to Kibana. Kibana autonomously queries Elasticsearch to generate visualisations from it. In the following we illustrate a selection of visualizations of vulnerability data:<\/p>\n<p style=\"text-align: center;\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-40916 size-full\" src=\"https:\/\/www.pentestfactory.de\/wp-content\/uploads\/10.png\" alt=\"\" width=\"893\" height=\"492\" title=\"\" srcset=\"https:\/\/www.pentestfactory.de\/wp-content\/uploads\/10.png 893w, https:\/\/www.pentestfactory.de\/wp-content\/uploads\/10-300x165.png 300w, https:\/\/www.pentestfactory.de\/wp-content\/uploads\/10-768x423.png 768w\" sizes=\"(max-width: 893px) 100vw, 893px\" \/><em>Figure: Amount of registered vulnerabilities per year<\/em><\/p>\n<p style=\"text-align: center;\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-40918 size-full\" src=\"https:\/\/www.pentestfactory.de\/wp-content\/uploads\/11.png\" alt=\"\" width=\"902\" height=\"566\" title=\"\" srcset=\"https:\/\/www.pentestfactory.de\/wp-content\/uploads\/11.png 902w, https:\/\/www.pentestfactory.de\/wp-content\/uploads\/11-300x188.png 300w, https:\/\/www.pentestfactory.de\/wp-content\/uploads\/11-768x482.png 768w\" sizes=\"(max-width: 902px) 100vw, 902px\" \/><em>Figure: Fractions of the respective risk severity groups per year<\/em><\/p>\n<p>We see great potential in using this data for real time statistics on our homepage to provide vulnerability trends which are updated on a daily basis.<\/p>\n<\/p>\n<h3>Outlook &#8211; Threat Intelligence and automatization<\/h3>\n<p>Another item on our CVE database roadmap is the implementation of a system that automatically notifies customers of new vulnerabilities, once they are released for a certain CPE identifier. Elasticsearch offers an extensive REST API that allows us to realize this task with the already implemented ELK stack.<\/p>\n<p>Currently we are working on implementing live statistics for our homepage. As soon as this milestone is complete, we will continue with the topic of &#8220;Threat Intelligence&#8221;. As you can see, we not only focus on the field of penetration testing here at Pentest Factory GmbH, but also have great interest in researching cybersecurity topics and extending our understanding, as well as our service line.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>Not only for penetration testing it is interesting to know, which vulnerabilities exist for a certain software product. Also from the perspective of an IT team it can be useful to quickly obtain information about an employed product version. So far various databases existed for these queries like e.g., https:\/\/nvd.nist.gov\/vuln\/search, https:\/\/cvedetails.com or https:\/\/snyk.io\/vuln However, during [&hellip;]<\/p>\n","protected":false},"author":6,"featured_media":40947,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[22],"tags":[32,33,31],"class_list":["post-41020","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-specialist-article","tag-cve-en","tag-nist-en","tag-vulnerability-database"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.pentestfactory.de\/en\/wp-json\/wp\/v2\/posts\/41020","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.pentestfactory.de\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.pentestfactory.de\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.pentestfactory.de\/en\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/www.pentestfactory.de\/en\/wp-json\/wp\/v2\/comments?post=41020"}],"version-history":[{"count":1,"href":"https:\/\/www.pentestfactory.de\/en\/wp-json\/wp\/v2\/posts\/41020\/revisions"}],"predecessor-version":[{"id":41021,"href":"https:\/\/www.pentestfactory.de\/en\/wp-json\/wp\/v2\/posts\/41020\/revisions\/41021"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.pentestfactory.de\/en\/wp-json\/wp\/v2\/media\/40947"}],"wp:attachment":[{"href":"https:\/\/www.pentestfactory.de\/en\/wp-json\/wp\/v2\/media?parent=41020"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.pentestfactory.de\/en\/wp-json\/wp\/v2\/categories?post=41020"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.pentestfactory.de\/en\/wp-json\/wp\/v2\/tags?post=41020"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}