Open source data mining is an advanced computer tool that analyzes large sets of information from multiple sources and detects patterns, trends, and relationships within the data. Future situations and behaviors can then be predicted (Gooch, 2003, ¶ 3).
Prior to the use of data mining, obtaining this data was labor intensive, expensive and tedious (Marks, 2011). Agencies must process these huge quantities of information to facilitate informed decision making and mission completion. Often they have limited resources with which to complete this task. Gooch describes a “volume challenge” in which “local, state, and federal agencies alike all struggle with an ever-increasing amount of information that far exceeds their ability to effectively analyze it in a timely fashion”(Gooch, 2003, ¶ 1). Open source data mining has been the answer to this problem.
Data mining allows the analysis of mass data in an effective and efficient manner (Gooch, 2003). It has been so successful for the U.S. Department of Homeland Security that its use has expanded greatly in recent years. This growth includes not only the amount of data collected, but also the types of data and personal information of citizens. One only has to turn the TV on to hear news reports describing encroachments on the privacy by the National Security Agency and the Internal Revenue Service.
In addition to terrorism related activities, the Department of Defense (DoD) has used data mining to investigate fraud within its own ranks (U.S. Department of Defense, 2004), to monitor drug safety (Buxbaum, 2008), and in tracking global technology advances (Marks, 2011). Data gleaned from analysis of the emergence of technologies around the globe allows government agencies to predict catastrophic events such as civil and political unrest, humanitarian emergencies, economic crises, and epidemics. With this knowledge, the government can maintain a strategic advantage over other countries (Marks, 2011).
The biggest negative factor with data mining is the potential for violation of privacy of American citizens. The information collected and analyzed extends beyond general pools of data unassociated with specific individuals. Extremely sensitive personal data records involving health, finances, vital statistics, and phone conversations. Before September 11, 2001, the government could not obtain this information without a search warrant. The Patriot Act of 2001 lifted many restrictions, enabling the Central Intelligence Agency and the Federal Bureau of Investigation to fight terrorism more successfully (Jenks, 2001). Privacy was sacrificed for security.
- Buxbaum, P. (2008, Dec 29). DOD awards contract for drug safety data-mining project. Retrieved January 28, 2014 from http://www.govhealthit.com/news/dod-awards-contract-drug-safety-data-mining-project
- Gooch, T. P. (2003, November 1). Data mining and value-added analysis. The FBI law enforcement bulletin. Retrieved January 28, 2014 from http://www.thefreelibrary.com/Data mining and value-added analysis.-a0111496582.
- Marks, J. (2011, September 27). Spy agency to use data mining to spot emerging technologies. Retrieved January 28, 2014 from http://www.nextgov.com/technology-news/tech-insider/2011/09/spy-agency-to-use-data-mining-to-spot-emerging-technologies/54874/print/
- Jenks, R. (December, 2001). The USA Patriot Act of 2001: a summary of the anti-terrorism law’s immigration-related provisions. Retrieved January 28, 2014 from http://www.cis.org/USAPatriotAct-ImmigrationRelatedProvisions
- U.S. Department of Defense, (2004, August 20). Data-Mining of DoD purchase cards leads to multiple indictments, release no. 806-04. Retrieved January 28,2014 from http://www.defense.gov/releases/release.aspx?releaseid=7655
