*
http://wattsupwiththat.com/2010/01/22/u ... ta-issues/
*
Parliament To Discuss CRU - FUSAG In East Anglia
Parliament To Discuss CRU - FUSAG In East Anglia
Engineering is the art of making what you want from what you can get at a profit.
Thanks for that. I'll likely make a submission, in regards the full disclosure of the subtleties of useage and manipulation of software in regards simulations and data sets.
It's been a bug bear of mine dating way back into other fields. When I was a Research Fellow 10 years ago I became very disillusioned with the way the 'research industry' had started treating simulations like black-box devices no more complicated than using a calibrated ruler, and thus with, effectively, almost non-existant disclosure of how a simulation was actually done [wherein papers were getting to the point of essentially saying "and software xyx.1 was used, as shown in the graph of diagram 1; conclusions, data set and xyz.1 simulations matched, so data set proved!..."].
A 'piece of research' must be able to be reproduced by someone else, and if it can't then it is of no academic use. So if some means has been adopted to control the flow of data into a simulation in a certain way [which is not unreasonable in itself, as long as it is clearly explained], this must be fully disclosed, else the result would not be repeatable. Consequently, it can be clearly seen that the many subtelites of data streaming that are employed in a complex simulation cannot be disclosed within a concise paper, so it is entirely reasonable for an academic to expect his or her verbatim day-to-day logs and comments to form part of that academic work and, thus, should have no issue with full disclosure, however it has occurred.
Anyone who has actually done complex research simulations and data handling knows full-well that one can induce subtelties in the results by nominally irrelevant variations in method, and even the type of computer can make a difference (try running any simulations requiring >a million steps on a 32 bit and 64 bit versions of the same software, and you'll see what I mean!).
It's been a bug bear of mine dating way back into other fields. When I was a Research Fellow 10 years ago I became very disillusioned with the way the 'research industry' had started treating simulations like black-box devices no more complicated than using a calibrated ruler, and thus with, effectively, almost non-existant disclosure of how a simulation was actually done [wherein papers were getting to the point of essentially saying "and software xyx.1 was used, as shown in the graph of diagram 1; conclusions, data set and xyz.1 simulations matched, so data set proved!..."].
A 'piece of research' must be able to be reproduced by someone else, and if it can't then it is of no academic use. So if some means has been adopted to control the flow of data into a simulation in a certain way [which is not unreasonable in itself, as long as it is clearly explained], this must be fully disclosed, else the result would not be repeatable. Consequently, it can be clearly seen that the many subtelites of data streaming that are employed in a complex simulation cannot be disclosed within a concise paper, so it is entirely reasonable for an academic to expect his or her verbatim day-to-day logs and comments to form part of that academic work and, thus, should have no issue with full disclosure, however it has occurred.
Anyone who has actually done complex research simulations and data handling knows full-well that one can induce subtelties in the results by nominally irrelevant variations in method, and even the type of computer can make a difference (try running any simulations requiring >a million steps on a 32 bit and 64 bit versions of the same software, and you'll see what I mean!).
-
Josh Cryer
- Posts: 526
- Joined: Sun Aug 31, 2008 7:19 am
Let's hope they don't handle it like they did homeopathy: http://www.skeptics.org.uk/showthread.p ... parliament
(The NHS in the UK distributes homeopath remedies with taxpayer money.)
I especially liked this article from a week or so back: http://www.thisisleicestershire.co.uk/n ... ticle.html
We can at least be proud that the UK has no better politicians than we do.
(The NHS in the UK distributes homeopath remedies with taxpayer money.)
I especially liked this article from a week or so back: http://www.thisisleicestershire.co.uk/n ... ticle.html
We can at least be proud that the UK has no better politicians than we do.
Science is what we have learned about how not to fool ourselves about the way the world is.
Some times big endian vs little endian can make a difference.Anyone who has actually done complex research simulations and data handling knows full-well that one can induce subtelties in the results by nominally irrelevant variations in method, and even the type of computer can make a difference (try running any simulations requiring >a million steps on a 32 bit and 64 bit versions of the same software, and you'll see what I mean!).
Engineering is the art of making what you want from what you can get at a profit.
-
Josh Cryer
- Posts: 526
- Joined: Sun Aug 31, 2008 7:19 am
NASA releases GISTEMP and Model3, it's the least they can do to release HadCRU3 or however it's spelled. Especially since a great deal of the code is used on ClimatePrediction.net and users donate their valuable electricity to running the model.
Science is what we have learned about how not to fool ourselves about the way the world is.
In support of this, even if the data preprocessing and model file are disclosed, the software to run these is still a black-box, and can't be analysed due to closed source "proprietary" methods - ie someone has fenced an area to make a buck. This is a vote for a requirement of open source software in scientific research. However it does take resources to produce good software.chrismb wrote:It's been a bug bear of mine dating way back into other fields. When I was a Research Fellow 10 years ago I became very disillusioned with the way the 'research industry' had started treating simulations like black-box devices no more complicated than using a calibrated ruler, and thus with, effectively, almost non-existant disclosure of how a simulation was actually done [wherein papers were getting to the point of essentially saying "and software xyx.1 was used, as shown in the graph of diagram 1; conclusions, data set and xyz.1 simulations matched, so data set proved!..."].
A 'piece of research' must be able to be reproduced by someone else, and if it can't then it is of no academic use. So if some means has been adopted to control the flow of data into a simulation in a certain way [which is not unreasonable in itself, as long as it is clearly explained], this must be fully disclosed, else the result would not be repeatable.
Governments are often thought of as good sponsors of "fundamental research". It would be good if development of simulation software was considered "fundamental research" and funded in that manner - for wide use and critical analysis. Not that one size fits all, but it might be efficient for different researchers to widely use the same software, and to be able to extend the corners of it as the see fit. [Edit: they don't necessarily need to do the programming themselves.]
Last edited by BenTC on Sat Jan 23, 2010 6:08 am, edited 1 time in total.
In theory there is no difference between theory and practice, but in practice there is.
-
Josh Cryer
- Posts: 526
- Joined: Sun Aug 31, 2008 7:19 am
-
kunkmiester
- Posts: 892
- Joined: Thu Mar 12, 2009 3:51 pm
- Contact:
I think propriety can be retained for research, there just has to be enough disclosure to get the same model and insert the data the same way, etc. This was also brought up about data sets--companies would gather the temp data, but would sell it rather than release it. Using data like this is fine, as long as you know exactly which data was used, so that another researcher can make sure he's using the same. Such companies might even offer a discount, since they would know that studies to verify the research will be done, and you're guaranteed a larger pool of customers. It's understandable for such data to be proprietary, but the computer models are not. I can't see many commercial uses for them besides selling them to researchers, so an open source model would work better all the way around.
Evil is evil, no matter how small
If I ever found a case like that I'd strongly suspect a bug in the code.MSimon wrote:Some times big endian vs little endian can make a difference.
Regarding 'open source' for simulation software: The source being open for study and criticism is vital for healthy science. Source covered by a 'copyleft' license is nice and useful, but not so vital.
Yes. If the data is output to your file (tape - memory stick - etc.) is big endian and the next computer to use it is little endian there is a bug if the answers are unreal.hanelyp wrote:If I ever found a case like that I'd strongly suspect a bug in the code.MSimon wrote:Some times big endian vs little endian can make a difference.
Regarding 'open source' for simulation software: The source being open for study and criticism is vital for healthy science. Source covered by a 'copyleft' license is nice and useful, but not so vital.
Engineering is the art of making what you want from what you can get at a profit.
-
Josh Cryer
- Posts: 526
- Joined: Sun Aug 31, 2008 7:19 am
This problem is effectively non-existent with competent programmers. The data formats are all ASCII anyway, text files. Very simple CSV or fixed width formats. Extraordinarily simple to read and parse.MSimon wrote:Yes. If the data is output to your file (tape - memory stick - etc.) is big endian and the next computer to use it is little endian there is a bug if the answers are unreal.
(Thank God none of them decided to use XML!)
Science is what we have learned about how not to fool ourselves about the way the world is.
Ian "Harry" Harris says the climate stuff was not done by competent programmers.competent programmers
http://www.floppingaces.net/2009/11/30/ ... must-read/
http://www.anenglishmanscastle.com/HARRY_READ_ME.txtAs another pundit said: this isn’t just the smoking gun pointing to the fraud of global warming, it’s a mushroom cloud!
From a commenter at Flopping Aces:…The file — 274 pages long — describes the efforts of a climatologist/programmer at the Climatic Research Unit (CRU) of the University of East Anglia to update a huge statistical database (11,000 files) of important climate data between 2006 and 2009.
The computer coding, along with the programmer’s apparently unsuccessful efforts to complete the project, involve data that are the foundation of the study of climate change — recordings from hundreds of weather stations around the world of temperature and precipitation measurements from 1901 to 2006, sun/cloud computer simulations, and the like.
…
The CRU at East Anglia University is considered by many as the world’s leading climate research agency. Here’s how CBSNews.com…’s Declan McCullagh describes its enormous impact on policymakers:
“In global warming circles, the CRU wields outsize influence: It claims the world’s largest temperature data set, and its work and mathematical models were incorporated into the United Nations Intergovernmental Panel on Climate Change’s 2007 report. The report … is what the Environmental Protection Agency acknowledged it ‘relies on most heavily’ when concluding carbon dioxide emissions endanger public health and should be regulated.”
As you read the programmer’s comments below, remember, this is only a fraction of what he says.
- “But what are all those monthly files? DON’T KNOW, UNDOCUMENTED. Wherever I look, there are data files, no info about what they are other than their names. And that’s useless …” (Page 17)
- “It’s botch after botch after botch.” (18)
- “The biggest immediate problem was the loss of an hour’s edits to the program, when the network died … no explanation from anyone, I hope it’s not a return to last year’s troubles … This surely is the worst project I’ve ever attempted. Eeeek.” (31)
- “Oh, GOD, if I could start this project again and actually argue the case for junking the inherited program suite.” (37)
- “… this should all have been rewritten from scratch a year ago!” (45)
- “Am I the first person to attempt to get the CRU databases in working order?!!” (47)
- “As far as I can see, this renders the (weather) station counts totally meaningless.” (57)
- “COBAR AIRPORT AWS (data from an Australian weather station) cannot start in 1962, it didn’t open until 1993!” (71)
- “What the hell is supposed to happen here? Oh yeah — there is no ’supposed,’ I can make it up. So I have : – )” (98)
- “You can’t imagine what this has cost me — to actually allow the operator to assign false WMO (World Meteorological Organization) codes!! But what else is there in such situations? Especially when dealing with a ‘Master’ database of dubious provenance …” (98)
- “So with a somewhat cynical shrug, I added the nuclear option — to match every WMO possible, and turn the rest into new stations … In other words what CRU usually do. It will allow bad databases to pass unnoticed, and good databases to become bad …” (98-9)
- “OH F— THIS. It’s Sunday evening, I’ve worked all weekend, and just when I thought it was done, I’m hitting yet another problem that’s based on the hopeless state of our databases.” (241).
- “This whole project is SUCH A MESS …” (266)
Are you getting it yet Josh?This is NOT science. A scientist reveals everything and dares you to break it. This is a scam.
Engineering is the art of making what you want from what you can get at a profit.
Shrug. Nothing ever comes of politicians investigating their policies.
Anyways, give me half of CRUs budget and I'll give you an equally plausible GCIM that says we're all going to freeze to death if we don't start pumping out CO2 and methane as fast as possible, and as a bonus I'll throw in another that says nothing we do makes any difference.
The whole thing is a yawn except to the ideologues who find it useful and the gullible who find it believable.
Anyways, give me half of CRUs budget and I'll give you an equally plausible GCIM that says we're all going to freeze to death if we don't start pumping out CO2 and methane as fast as possible, and as a bonus I'll throw in another that says nothing we do makes any difference.
The whole thing is a yawn except to the ideologues who find it useful and the gullible who find it believable.