Princeton Researchers Say Feds Need Data Standard 49
dcblogs writes "The federal government's data-sharing efforts are a mess, and if Barack Obama really wants a useful 'Google for government,' he would have to set the government's vast amount of data free by exposing it and ensuring it complies to standards. Once that happens, commercial sites, aggregators, bloggers and everyone else will be able to access it, use it and transform it, argue a group of Princeton researchers (follow Download link for full PDF)."
Rank and filed. (Score:1, Interesting)
I'm not certain I would agree with setting it all free. However it all does need to be standardized.
As for the rest. WEB 2.0!
Re: (Score:2)
And no one said set it ALL free. But to set what data that citizens should be able to access free, it needs to be in an open format that anyone can access.
Add them to the buying spree. (Score:5, Funny)
Well, so far the government has bought parts of Bear Stearns and AIG. Maybe it's time they diversify into some technology companies like Google? Hell, let's buy them too!
Re: (Score:1, Flamebait)
Well... the EU, Australia and Japan/Korea would shit in their pants (because they would become the non-believing axis or evil,
and China and Cuba would come all over the place from having a new ally.
Everyone else would not care and say "same old shit, different day".
On the other hand, as the EU imitates the US in every aspect they can, maybe they would simply become communist too, just in a more bureaucratic way... Hey, in EU's government, because of their bureaucracy, communism could actually work. ;)
Re:Add them to the buying spree. (Score:4, Insightful)
In the meantime, if you want the government to produce useful data, don't insist that they standardize. Government employees are not particularly good at standardization, and if publishing requirements slow them down, then they just won't release data. Free, standard, and available are all possible, as long as you only want 2.
Re:Add them to the buying spree. (Score:5, Interesting)
Re: (Score:1, Insightful)
I totally disagree. Who cares what format the data is in? The real challenge is obtaining data that you need in the first place. This is *NOT* about pedantic word document format wars. Its about access to raw datasets.
You can already download thousands of large datasets in CSV form via FTP from many government information sites. The raw data needs work to process and generate user readable reports from but anyone with half a clue processing data is easily able to do that. Format has always been a non-is
Librarians (Score:3, Interesting)
The government could hire librarians to organize the data. This is are a group of people highly trained in how to take large quantities of non standard data and organize it in a way that people can find what they want.
Re: (Score:2)
Creating large CRM systems is not free. Very little can be done other than "dump it online" if the cost has to be kept very low.
Re: (Score:2)
You can't get the fed buying google, only losses get socialized
Sure you can, it just takes a while to set it up. Require google to feature sites that don't bring them ad revenue (Not that well meaning government would ever meddle in a market that was working reasonably well otherwise just for social engineering [ffiec.gov]) and then when Google fails, blame it on their own greed and socialize them.
Re:Add them to the buying spree. (Score:4, Insightful)
Re: (Score:1)
GOXML (Score:5, Funny)
I hear that Microsoft is already working on the problem with their proposed "Government Open XML" standard. This should not be confused with GOXMLb ("Google Open XML beta") because Microsoft would never try to confuse people on such an issue.
It is going up for ISO vote next week. Be there*.
(*) it will be very profitable for you to "be there".... nudge, nudge... wink, wink...
Re: (Score:2)
The problem that the U.S.Government faces is the lack of understanding about all things data. Obviously, there is secret data, and public data. The secret stuff we will never see, but the public stuff? Now THAT's worth considering. XML is an excellent format for data, and could work very well hiding information. But someone is going to have to convert written documents to XML; "Mind Deadening" would be a most polite term. But one of the nice things that could be applied here is the number of U.S.Citiz
Looks like a job for Microsoft! (Score:4, Insightful)
What you need is not one [set of] standard(s) but one vendor controlling and maintaining those standards... they know what is best for all of us because they are paid professionals, not hack hobbyists.
(Yes! I am kidding!!)
Re: (Score:3, Insightful)
You're kidding ... but Microsoft isn't.
It makes me nervous when people say things like this line from TFA: "Private actors, either nonprofit or commercial, are better suited to deliver government information to citizens ..." Uh, no, governments are better suited to deliver information about themselves, and no matter how bureaucratic or obstructionist the US government may be it's still more open and credible, on a dollar-for-dollar basis, than a lot of the "private actors" who would just looove to charge
Re: (Score:2)
Re:Welcome to the "duh" department (Score:4, Insightful)
Re: (Score:3, Insightful)
only if you take the long route. the quickest way to get the government to listen is with an unattended briefcase full of cash.
that is, if you don't already have former board members in the White House.
Shakespeare:To share or not to share? (Score:3, Interesting)
One thought has occurred to me as part of this "sharing". Privacy and the other is Security.
Re:Shakespeare:To share or not to share? (Score:5, Informative)
Re: (Score:3, Informative)
"The District of Columbia is far ahead of its federal government overlord in bringing data to standard XML formats and RSS-enabling it. DC's government has what it calls a "data catalog" offering live data feeds of crime reports, construction reports, building permits and many other types of information. "
I can see some that if not screened carefully could cause problems in this "sharing" environment.
Also don't forget there have been examples were citizen information has accidentally been leaked. Soon retra
Re: (Score:2)
"To share" is a GoodThing(TM), but my brain keeps showing me pictures of the Vogon destructor fleet.
Re: (Score:2, Funny)
-- A Usenet
Re: (Score:1)
We are not talking about the government sharing data on individual citizens or on military secrets. We are talking about things involving government spending, contracts, loans, grants, etc. Things that citizens should have access to, but have trouble organizing.
Since the government spends, contracts, lends, and grants to individual citizens and to military contractors, we are in fact talking about sharing sensitive data. That's why it needs to be carefully reviewed before publishing it. SSNs have already appeared on the web [fedspending.org] as a result of efforts to share government grant and contract data.
Re: (Score:2)
It's in the best interest of the government, it's employees, and contractors that this information is as hard to find as possible. The last thing they want is for the people to be able to find information on what is really going on.
Imagine a beowulf cluster of lobbyists (Score:3, Insightful)
Remember the good old days, when transparency in government could be safely considered a good thing?
Generally, I'm still for it. Absolutely we need transparency in our government, and anything that brings us closer to point-and-click convenience over what we have now (FOIA requests left behind the radiator for 9–18 months to age and mellow) is for the best.
Furthermore, an open, accessible standard (i.e. no copyrighted DTDs, and I'm looking at you, Microsoft) will allow government resources to be brought together in interesting and inspiring ways. You know all those Facebook apps and Google Maps mashups? Imagine those applied to governance. The idea behind them is to put information together in new and interesting ways. If not only those in government, but the citizenry, can create government hacks like that, there would be great benefit.
Now let's talk hazards.
When was the last time you published your name and address online? See any good uses of microformats [microformats.org] on any major sites lately? That's because there are some people on the Internet who are <sarcasm class="churchlady">not so nice</sarcasm>, and might willingly abuse whatever information they can find. The "government hack" alluded to above is an invitation to abuse. And we really can't afford to put government in that kind of position.
Another consideration, and I've stated this before, is that a wide line must be maintained between security and transparency. Security means that everything that must be kept secret is really kept secret. Transparency means that everything that doesn't have to be secure is made available somehow. If things aren't secured, the government becomes ineffectual and even detrimental. If things aren't kept transparent, the government itself can become abusive. A freely searchable infrastructure would make the transparency all that much more powerful, and make any breaches in security that much more severe.
Good advice doesn't always have good result (Score:4, Insightful)
I use lots of government supplied data in my work, and one constant has always been that the more work the agency does to make the data easily available, the harder the data are to use. Spreadsheets get posted with labels and data mixed, because that looks better in print. Spreadsheets get posted as PDFs, because that looks better in print. Footnotes and other textual material is mixed into numeric fields, because that is the way the material will be published in hardcopy. etc etc etc.
Databases get posted to the web with "interfaces" that allow single rows to be downloaded, but require months of screen scraping to get the entire database. Databases get released with (windows-only, of course) software with the same effect. etc etc etc
The reason is mostly that agencies want to discourage outside analysis of the data - they would prefer to avoid inconsistent messages getting to OMB or congress.
Re: (Score:2)
Really, I just think there's so much data out there that it's hard to do this effectively without compromising things. I'm not sure what the solution is, but letting a search engine go wild on the government's data seems possibly worse than the current state of affairs.
Various Departments.... (Score:1)
you mean like this for example? (Score:2)
http://www.fgdc.gov/standards [fgdc.gov]
Princeton PhDs says so, we must do (Score:2)
Come on, a bunch of Princeton researchers, after spending $X millions of grants from the US gov't over 5 years now says we need a data standard?
Dublin Core?
FEA? (Federal Enterprise Architecture, the other DRM)
OAIS?
And talk to anyone in gov't IT today on fed data problems, and they'll give you better info on how to solve the data issues vs. these researchers. Note to the Princeton researchers: stick to solving the semantic web problems--cause that's something we can ignore for the next 10 yrs.
And Google for
Standard. (Score:2)
ensuring it complies to standards
Hopefully he knows the meaning of a TRUE [wikipedia.org] standard, as opposed to the other [wikipedia.org] kind!
WTF...WTI - DCBlogs is wrong or the PRs are idiots (Score:2)
"The federal government's data-sharing efforts are a mess" This is very well known and real PRs would not need to state the obvious.
We The People really wants a useful government, but continue to elect tent-revivalist and pick-pocket politicians (most not all are in on the $7B scam).
Set the government's vast amount of data free is total bullshit.
Globally exposing Gov/Mil data/content IS NOT REQUIRED to ensure web-services/SOA and data/content complies with "Open" standards like UTF, ODT, PNG/SVG..., syntax
I'm surprised (Score:2)
follow Download link for full PDF
they didn't publish this using Silverlight.
Well, its already happening in some places (Score:2, Informative)
The United States Army uses 'PureEdge' which i guess was replaced by IBM with 'Lotus Forms' as there is no canonical link to the software anymore. Its an XML based form system. Its not really used in any standard way, other than electronically saving forms, and filling stuff in before printing the forms. It could though, because the Army, at least, does little to no documentation that isn't on some kind of standardized form. Now that the forms are machine parsable, I can definitely see the fed adoption some
CSV eXtended (Score:2)
What would be a good start would be to standardise the publication of tabular data, for example population statistics, with ways of defining column types, data types and units whilst retaining a tabular structure instead of bastardising the tree-structure of XML. I guess we could take CSV, add a couple of header blocks and call it Extended CSV. Though it'd need an X in it to sound 21st century... so how about CSVX?
If anyone googles that my web site will go down in flames...