Organizations worldwide are utilizing big data practices to improve process and performance. It would seem only natural that one of the agencies collecting the largest amounts of data would make it a priority to do the same. While the government has implemented big data practices in some areas in the state and federal levels, it’s not used widely enough to see the big changes we’d hope for.
Data being collected
When you think about what data is being collected about you and your neighbor, it’s quite overwhelming. For instance, the NSA compiles loads of phone data, surveillance footage, email and Facebook posts, internet traffic trends, etc. And that’s just the NSA. Other agencies compile information like local census details about the population, school children’s test scores in relation to their socioeconomic standing and race, political trends between districts…the list goes on.
Why is this data important?
Armed with information, we’re able to make informed decisions about how to improve processes and work toward the overall betterment of our communities. For instance, having the data which proves school children who commute for 30 minutes or longer to school each day perform poorly on standardized testing could spawn action to rectify that situation. Note, this is not a documented fact, purely an example.
How government is using big data currently
On the Federal level, the Obama administration appointed DJ Patel as the first US Chief Data Scientist and committed $200 million to big data practices. In his position, Mr. Patel will aim to help shape policies and practices to keep the U.S.A. at the top of the technology leaderboard.
More than 60% of government agencies have used big data to reduce costs and improve efficiencies. Most of these agencies are also utilizing big data to improve the security of their IT practices. Keeping a close eye on advanced threats through the automation of tracking inconsistencies in machine data, they’ve been able to stop attacks before they cause any issues.
Future of big data in government
The list of benefits from deploying big data software throughout the government are endless especially considering that data scientists across the globe are constantly improving performance and dreaming up new data streams to track. Where we’d see the most benefit would be in increasing private sector employment, improving high school graduation rates, and enhancements in public health programs. Predictive analytics seeking to improve these very areas have already been in action in Indiana. Governor Mike Pence saw to it that big data is being utilized in agencies across the state.
So, what’s stopping them?
While big data seems like a no brainer, the government doesn’t have the freedom nor the resources of the private sector which makes rolling out programs like this kind of a pain. Lack of staffing resources is a major roadblock preventing them from modernizing their data usage. Finding the necessary talent capable of running these programs has been a challenge. Furthermore, many agencies simply don’t have the infrastructure to support big data software. In many cases, a rip and replace infrastructure refresh would be necessary to be sure their storage, compute and networking infrastructures could handle the data load.
While the majority of public sector agencies aren’t ready for big data today, the good news is they’re thinking about it and realize the benefits it would provide not only to them but also to our communities.