Annual Coverity Scan Report Finds Open Source and Proprietary Software Quality Better than Industry Average for Second Consecutive Year. Coverity
Annual Coverity Scan Report Finds Open Source and Proprietary Software Quality Better than Industry Average for Second Consecutive Year [link]Paper  abstract   bibtex   
[Excerpt] The Coverity Scan Open Source Report has become a widely accepted standard for measuring the state of open source quality. During the past eight years, the Coverity Scan service has analyzed several hundreds of millions of lines of code from more than 1,500 open source projects - including C/C++ projects such as NetBSD, FreeBSD, LibreOffice and Linux, and Java projects such as Apache Hadoop, HBase and Cassandra. The Scan service has helped developers find and fix more than 94,000 defects since 2006. Nearly 50,000 defects were fixed in 2013 alone - the largest single number of defects fixed in a single year by Scan users. More than 11,000 of these defects were fixed by the four largest projects in the service: NetBSD, FreeBSD, LibreOffice and Linux. [Key findings] Key findings in the 2013 report include: [Open source code quality surpasses proprietary code quality in C/C++ projects] Defect density (defects per 1,000 lines of software code) is a commonly used measurement for software quality, and a defect density of 1.0 is considered the accepted industry standard for good quality software. Coverity's analysis found an average defect density of .59 for open source C/C++ projects that leverage the Scan service, compared to an average defect density of .72 for proprietary C/C++ code developed for enterprise projects. In 2013, code quality of open source projects using the Scan service surpassed that of proprietary projects at all code base sizes, which further highlights the open source community's strong commitment to development testing. [Linux continues to be a benchmark for open source quality] By leveraging the Scan service, Linux has reduced the average time to fix a newly detected defect from 122 days to just 6 days. Since the original Coverity Scan Report in 2008, scanned versions of Linux have consistently achieved a defect density of less than 1.0. In 2013, Coverity scanned more than 8.5 million lines of Linux code and found a defect density of .61. [C/C++ developers fixed more high-impact defects] The Coverity analysis found that developers contributing to open source Java projects are not fixing as many high-impact defects as developers contributing to open source C/C++ projects. Java project developers participating in the Scan service only fixed 13 percent of the identified resource leaks, whereas participating C/C++ developers fixed 46 percent. This could be caused in part by a false sense of security within the Java programming community, due to protections built into the language, such as garbage collection. However, garbage collection can be unpredictable and cannot address system resources so these projects are at risk. [HBase serves as benchmark for Java projects] Coverity analyzed more than 8 million lines of code from 100 open source Java projects, including popular Big Data projects Apache Hadoop 2.3 (320,000 lines of code), Apache HBase (487,000 lines of code) and Apache Cassandra (345,000 lines of code). Since joining the Scan service in August 2013, Apache HBase - which is Hadoop's database - fixed more than 220 defects, including a much higher percentage of resource leaks compared to other Java projects in the Scan service (i.e., 66 percent for HBase compared to 13 percent on average for other projects).
@article{coverityAnnualCoverityScan2014,
  title = {Annual {{Coverity}} Scan Report Finds Open Source and Proprietary Software Quality Better than Industry Average for Second Consecutive Year},
  author = {{Coverity}},
  date = {2014},
  url = {http://mfkp.org/INRMM/article/13460170},
  abstract = {[Excerpt] The Coverity Scan Open Source Report has become a widely accepted standard for measuring the state of open source quality. During the past eight years, the Coverity Scan service has analyzed several hundreds of millions of lines of code from more than 1,500 open source projects - including C/C++ projects such as NetBSD, FreeBSD, LibreOffice and Linux, and Java projects such as Apache Hadoop, HBase and Cassandra. The Scan service has helped developers find and fix more than 94,000 defects since 2006. Nearly 50,000 defects were fixed in 2013 alone - the largest single number of defects fixed in a single year by Scan users. More than 11,000 of these defects were fixed by the four largest projects in the service: NetBSD, FreeBSD, LibreOffice and Linux.

[Key findings] Key findings in the 2013 report include:

[Open source code quality surpasses proprietary code quality in C/C++ projects] Defect density (defects per 1,000 lines of software code) is a commonly used measurement for software quality, and a defect density of 1.0 is considered the accepted industry standard for good quality software. Coverity's analysis found an average defect density of .59 for open source C/C++ projects that leverage the Scan service, compared to an average defect density of .72 for proprietary C/C++ code developed for enterprise projects. In 2013, code quality of open source projects using the Scan service surpassed that of proprietary projects at all code base sizes, which further highlights the open source community's strong commitment to development testing.

[Linux continues to be a benchmark for open source quality] By leveraging the Scan service, Linux has reduced the average time to fix a newly detected defect from 122 days to just 6 days. Since the original Coverity Scan Report in 2008, scanned versions of Linux have consistently achieved a defect density of less than 1.0. In 2013, Coverity scanned more than 8.5 million lines of Linux code and found a defect density of .61.

[C/C++ developers fixed more high-impact defects] The Coverity analysis found that developers contributing to open source Java projects are not fixing as many high-impact defects as developers contributing to open source C/C++ projects. Java project developers participating in the Scan service only fixed 13 percent of the identified resource leaks, whereas participating C/C++ developers fixed 46 percent. This could be caused in part by a false sense of security within the Java programming community, due to protections built into the language, such as garbage collection. However, garbage collection can be unpredictable and cannot address system resources so these projects are at risk.

[HBase serves as benchmark for Java projects] Coverity analyzed more than 8 million lines of code from 100 open source Java projects, including popular Big Data projects Apache Hadoop 2.3 (320,000 lines of code), Apache HBase (487,000 lines of code) and Apache Cassandra (345,000 lines of code). Since joining the Scan service in August 2013, Apache HBase - which is Hadoop's database - fixed more than 220 defects, including a much higher percentage of resource leaks compared to other Java projects in the Scan service (i.e., 66 percent for HBase compared to 13 percent on average for other projects).},
  keywords = {*imported-from-citeulike-INRMM,~INRMM-MiD:c-13460170,~to-add-stable-URL,comparison,free-software,software-errors,software-quality}
}

Downloads: 0