Information about http://www.oceansites.org/docs/OTS2008/CoriolisGDAC_status.pdf
Coriolis GDAC status …
Tags: checker, co data, commitments, fil, gdacs, keo, loco, mbari, pi, pirata, proposal, tao, vienne,Pages: 10
Language: english
Created: Tue Jul 22 09:55:05 2008
Display cached document
Page 1
Page 2
Page 3
Page 4
Page 5
Page 6
Page 7
Page 8
Page 9
Page 10
Coriolis GDAC status
ement Vienne 10/04/2008
8 Proposal of Commitments February 2006
Coriolis GDAC status April 2008
e
OceanSites format checker
Example of check : ESTOC file
data manage
H h h k
How to use the checker ?
KEO example
How to provide files to OceanSites GDACs ?
Data collect
OceanSites index file
O Sit i d fil
nSites
Ocean
1
Proposal of Commitments - February 2006
ement Vienne 10/04/2008
8
PMEL : ok t provide K & th TAO/Pirata data in oceansites format
k to id Keo then TAO/Pi t d t i it f t
MBARI could volunteer to be one of the GDACs... OK to provide some of the MBARI
data in OceanSites
e
CDIAC OK to provide COČ data in OceanSITES format
UCSB ok to provide some of the data in OceanSITES format (BTM)
LOCO : willing to provide some of the data in OCANSITES format
OCO g o p o de so e o e da a OC S S o a
data manage
CCHDO: could provide ship occupied CTD repeated stations. Will provide tool to
convert in Oceansites format
Jamstec: while their system is set up will provide Triton data in OceanSites format in
format.
2007
NOC: need to include ADCP and current meter data... Will continue the way they
started with Coriolis
IPRC:: Can Act like a DAC for some orphan OCEANSites DAC .IPRC could pursue work
nSites
on DAPPER and LAS to give visibility of the network..
HAWAII: HOTS: CTD data , WHOTS with WHOI ( RT and DM)
Ocean
WHOI : has already started and will continue
BATS : CTD data like for HOTS
Coriolis volunteers to host one of the GDAC
NOAA is interested in hosting one GDAC in 2007
2
Coriolis GDAC status April 2008
PMEL : ok to provide Keo & then TAO/Pirata data in oceansites format
ement Vienne 10/04/2008
8
MBARI could volunteer to be one of the GDACs... OK to provide some of the MBARI
data in OceanSites
CDIAC OK to provide COČ data in OceanSITES format
e
UCSB ok to provide some of the data in OceanSITES format (BTM)
LOCO : willing to provide some of the data in OCANSITES format
data manage
CCHDO: could provide ship occupied CTD repeated stations. Will provide tool to
convert in Oceansites format
Jamstec: while their system is set up will provide Triton data in OceanSites format. in
y p p
2007
NOC: need to include ADCP and current meter data... Will continue the way they
started with Coriolis
IPRC:: Can Act like a DAC for some orphan OCEANSites DAC .IPRC could pursue work
nSites
on DAPPER and LAS to give visibility of the network..
HAWAII: HOTS: CTD data , WHOTS with WHOI ( RT and DM)
Ocean
WHOI : has already started and will continue
BATS : CTD data like for HOTS
Coriolis volunteers to host one of the GDAC
NOAA is interested in hosting one GDAC in 2007
3
Coriolis GDAC status April 2008
ement Vienne 10/04/2008
8 Files already collected by OceanSites GDAC :
· ANIMATE : 13 files (5 CIS-1, 2 CIS-2, 1 ESTOC-1, 3 PAP-1 and 2 PAP-2)
e
· BTM : 1 file
· KEO : 4 files (4 KEO32N145E)
· MOVE : 17 files (5 V404, 1 V404_2005, 1 V404_2006, 5 V405 and 5 V406)
V404 V404 2005 V404 2006
data manage
· NTAS : 2 files
· STRATUS : 4 files
· TAO : 37 files (11 moorings, including carbon data)
nSites
Ocean
4
OceanSites format checker
ement Vienne 10/04/2008
8 An O
OceanSites format checker is available.
f h k l bl
Example of check on file OS_ESTOC-1_200711_TSC.nc:
control.csh oceansites/OS_ESTOC-1_200711_TSC.nc
e
? l i "1 0"?
CO-03-08-03
Control file data format
Wed Apr 09 10:17:00 MEST 2008
oceansites/OS_ESTOC-1_200711_TSC.nc
/ /
data manage
oceansites_1.0.xml
OceanSITES data
1.0
The mandatory global attribute "comment" is missing
The mandatory attribute of variable "DATE_UPDATE:long_name" is missing
The mandatory attribute of variable "DATE SOURCE:long name" is missing
DATE_SOURCE:long_name
The value of the attribute of variable "DATA_RESTRICTIONS:comment" is not correct : "Restriction on use for these
data" expected
The value of the attribute of variable "CITATION:comment" is not correct : "Restriction on use for these data"
expected
The value of the attribute of variable "DEPH:long_name" is not correct : "depth of each measurement"
expected
p /
The mandatory attribute of variable "DEPH:epic_code" is missing
The value of the attribute of variable "PRES:long_name" is not correct : "sea water pressure" expected
nSites
The value of the attribute of variable "PSAL:standard_name" is not correct : "sea_water_salinity" expected
The mandatory attribute of variable "PSAL:QC_indicator" is missing
The value of the attribute of variable "TEMP:long_name" is not correct : "sea water temperature" expected
The mandatory attribute of variable "TEMP:QC indicator" is missing
TEMP:QC_indicator
Ocean
The value of the attribute of variable "TEMP_QC:long_name" is not correct : "quality flag" expected
The mandatory attribute of variable "TEMP_QC:conventions" is missing
no
5
OceanSites format checker
Download OceanSites format checker at :
D l dO Sit f t h k t
ement Vienne 10/04/2008
8
http://www.ifremer.fr/co/etc/oceansites/checker/checker.tar.gz
checker.tar.gz contains directories, 1 control.bat and 1 control.csh (it works on linux and windows).
e
Check configuration variables in controle.csh or control.bat :
You have to change application path.
You may have to change the Java environment variables.
data manage
Launch the checker : controle.csh OceanSites_file
That gives you an xml report.
This format checker is based on OceanSITES user manual v1.0 (from
http://www.oceansites.org/docs/oceansites-user-manual.pdf)
The h k
Th checker works only if DATA_TYPE and FORMAT_VERSION are declared as variables.
k l DATA TYPE d FORMAT VERSION d l d i bl
In format v1.0 :
nSites
- DATA_TYPE = "OceanSITES data" ;
- FORMAT_VERSION = "1.0 " ;
Ocean
Only ANIMATE moorings (CIS, PAP and ESTOC) pass the checker.
Thanks to ANIMATE files, we found errors that highlight some inconsistencies in User Manual and we
updated it for the next version (v1.1).
After adding DATA_TYPE and FORMAT_ VERSION variables, STRATUS, MOVE, KEO and TAO files are
nearly in OceanSites format v1.0.
6
OceanSites format checker
ement Vienne 10/04/2008
8 Example of check on file OS_KEO32N145E_200406_TSVM_dy.nc
After adding mandatory variables for identify the rules file :
· DATA_TYPE = "OceanSITES data "
· FORMAT VERSION = "1 0"
FORMAT_VERSION 1.0
e
control.csh oceansites/OS_KEO32N145E_200406_TSVM_dy.nc
CO-03-08-03
Control file data format
data manage
Fri Apr 04 13:47:29 MEST 2008
oceansites/OS_KEO32N145E_200406_TSVM_dy.nc
oceansites_1.0.xml
OceanSITES KEO in-situ data
The mandatory dimension "TIME" is missing.
The mandatory dimension "LEVEL" is missing.
y g /
The mandatory dimension "STRING256" is missing.
The mandatory dimension "STRING64" is missing.
The mandatory dimension "STRING32" is missing.
The mandatory dimension "STRING14" is missing.
The mandatory dimension "STRING8" is missing.
The mandatory dimension "STRING4" is missing.
The mandatory global attribute "creation_date" is missing
Th d l b l ib " i d "i i i /
The mandatory global attribute "comment" is missing
nSites
The mandatory variable "name QC_MANUAL" is missing
The mandatory variable "name DATE_CREATION" is missing
The mandatory variable "name DATE_UPDATE" is missing
The mandatory variable "name DATE_SOURCE" is missing
The mandatory variable "name DATA_RESTRICTIONS is missing
name DATA RESTRICTIONS"
Ocean
The mandatory variable "name CITATION" is missing
The mandatory variable "name PLATFORM_CODE" is missing
The mandatory variable "name SITE_CODE" is missing
The mandatory variable "name WMO_PLATFORM_CODE" is missing
The mandatory variable "name PROJECT_NAME" is missing
The mandatory variable "name PI NAME" is missing
name PI_NAME
The mandatory variable "name DATA_CENTRE" is missing
The mandatory variable "name TIME" is missing
The mandatory variable "name DEPH, dimension_count 1" is missing
no
7
How to provide files to OceanSites GDACs ?
ement Vienne 10/04/2008
8 There are two methods to provide files to OceanSites GDACs:
h h d d fl
, y
First method, GDACs download files from your FTP
e
All you have to do is to give GDACs some information bellow:
ftp address
data manage
L i
Login
Password
Directory
y
Second method, GDACs give you an FTP access
nSites
Then GDACs will regularly collect your files.
Ocean
8
Data collect
ement Vienne 10/04/2008
8 Currently, files provided to OceanSites GDACs are collected twice a day.
l fl d d ll d d
y y
Files will be automatically tested by the OceanSites format checker.
e
format is checked
some metadata will be checked (for example : site code existence)
data manage
A file no compliant will be moved in a "rejected" directory.
All files accepted are distributed on OceanSites FTP.
An i d fil "
A index file "oceansites_index.txt" is updated in the same time.
it i d t t" i d t d i th ti
nSites
Ocean
9
OceanSites index file
oceansites_index.txt
oceansites index txt
ement Vienne 10/04/2008
8
# OceanSites FTP GLOBAL INDEX
# ftp://ftp.ifremer.fr/ifremer/oceansites
# Contact : http://www.oceansites.org/OceanSITES/index.html
# Creation date DD/MM/YYYY HH:MI:SS 30/10/2007 18:37:46
e
#
#FILE,DATE_UPDATE,SIZE (bytes)
TAO/0n170w/OS_0n170w_SW_LW_2m.nc,Fri Apr 4 07:34:26 2008,48352652
TAO/0n165e/OS_0n165e_SW_LW_2m.nc,Fri Apr 4 07:16:19 2008,90943028
TAO/0n140w/OS_0n140w_TSVM_dy.nc,Sun Apr 6 20:48:52 2008,5843736
ANIMATE/CIS/OS_CIS-2_200308_TSC.nc,Fri Jan 4 20:47:27 2008,978108
data manage
TAO/0n140w/OS_0n140w_S_hr.nc,Fri Apr 4 06 55 48 2008 20492988
O/0 140 /OS 0 140 S h i 06:55:48 2008,20492988
TAO/0n165e/OS_0n165e_S_hr.nc,Fri Apr 4 07:19:14 2008,36832444
TAO/0n170w/OS_0n170w_TVM_10m.nc,Fri Apr 4 07:42:23 2008,95235500
ANIMATE/CIS/OS_CIS_200509_TSC.nc,Thu Nov 29 20:47:55 2007,7718064
TAO/0n170w/OS_0n170w_S_hr.nc,Fri Apr 4 07:34:39 2008,1580668
ANIMATE/CIS/OS_CIS-1_200409_TSC.nc,Fri Jan 4 20:47:17 2008,8244220
TAO/0n80.5e/OS_0n80.5e_TSVM_dy.nc,Sun Apr 6 20:47:17 2008,1900264
ANIMATE/PAP/OS_PAP-1_200307_TSC.nc,Fri Dec 21 20:47:11 2007,1955160
ANIMATE/ESTOC/OS_ESTOC-1_200711_TSC.nc,Thu Mar 27 20:47:03 2008,49936
TAO/10s10w/OS_10s10w_TVM_10m.nc,Fri Apr 4 06:16:10 2008,100938772
Next Corrections :
TAO/0n165e/OS_0n165e_TVM_10m.nc,Fri A
TAO/0 165 /OS 0 165 TVM 10 F i Apr 4 07 30 37 2008 145510272
07:30:37 2008,145510272
ANIMATE/CIS/OS_CIS_200707_TSC.nc,Fri Nov 30 06:47:12 2007,195168
TAO/10s10w/OS_10s10w_S_hr.nc,Fri Apr 4 06:08:12 2008,5862760
TAO/0n140w/OS_0n140w_SW_LW_2m.nc,Fri Apr 4 06:54:09 2008,88901108
TAO/0n23w/OS_0n23w_TVM_10m.nc,Fri Apr 4 06:00:24 2008,84710756
ANIMATE/CIS/OS_CIS-1_200707_TSC.nc,Fri Jan 4 20:47:23 2008,195172
ANIMATE/PAP/OS_PAP-1_200406_TSC.nc,Tue Mar 11 20:47:05 2008,1029872 · Change format of the collect date
ANIMATE/PAP/OS_PAP-2_200406_TSC.nc,Tue Mar 11 20:47:10 2008,8483588
· Add date update of the file
p
TAO/0n165e/OS_0n165e_SW_2m.nc,Tue Nov 6 07:08:45 2007,53668408
TAO/0n23w/OS_0n23w_S_hr.nc,Fri Apr 4 05:53:45 2008,6123204
ANIMATE/CIS/OS_CIS-1_200308_TSC.nc,Fri J
ANIMATE/CIS/OS CIS 1 200308 TSC F i Jan 4 20 47 09 2008 357692
20:47:09 2008,357692
ANIMATE/CIS/OS_CIS_200409_TSC.nc,Thu Nov 29 20:47:41 2007,8244216
· Sort alphabetically observatory and moorings
ANIMATE/CIS/OS_CIS_200405_TSC.nc,Thu Nov 29 20:47:13 2007,2842512
nSites
TAO/0n170w/OS_0n170w_TSVM_dy.nc,Sun Apr 6 20:49:35 2008,2976972
ANIMATE/PAP/OS_PAP-2_200311_TSC.nc,Wed Jan 2 20:47:11 2008,4775764
TAO/0n165e/OS_0n165e_TSVM_dy.nc,Sun Apr 6 20:49:19 2008,5172204
TAO/15n38w/OS_15n38w_TVM_10m.nc,Fri Apr 4 06:28:01 2008,89274164
ANIMATE/CIS/OS_CIS_200208_TSC.nc,Thu Nov 29 20:47:11 2007,3617668
TAO/15n38w/OS_15n38w_S_hr.nc,Fri Apr 4 06:20:57 2008,5245036
TAO/15n38w/OS_15n38w_TSVM_dy.nc,Sun Apr 6 20:47:42 2008,1233276
Ocean
ANIMATE/CIS/OS_CIS-1_200509_TSC.nc,Fri J
ANIMATE/CIS/OS CIS 1 200509 TSC F i Jan 4 20 47 22 2008 7718068
20:47:22 2008,7718068
TAO/0n23w/OS_0n23w_SW_LW_2m.nc,Fri Apr 4 05:53:14 2008,77784776
TAO/0n110w/OS_0n110w_S_hr.nc,Fri Apr 4 06:35:01 2008,8441792
ANIMATE/ESTOC/OS_ESTOC-1_200712_TSC.nc,Tue Mar 11 06:47:10 2008,34480
TAO/0n23w/OS_0n23w_TSVM_dy.nc,Sun Apr 6 20:47:25 2008,1101088
ANIMATE/CIS/OS_CIS-2_200208_TSC.nc,Fri Jan 4 20:47:26 2008,3617672
KEO/KEO32N145E/OS_KEO32N145E_200406_TSVM_dy.nc,Sun Apr 6 20:49:44 2008,1070508
ANIMATE/CIS/OS_CIS-1_200405_TSC.nc,Fri Jan 4 20:47:11 2008,2842516
TAO/10s10w/OS_10s10w_TSVM_dy.nc,Sun Apr 6 20:47:33 2008,1278868
TAO/0n110w/OS_0n110w_TSVM_dy.nc,Sun Apr 6 20:48:21 2008,7315972
TAO/0n110w/OS_0n110w_SW_LW_2m.nc,Fri
TAO/0n110w/OS 0n110w SW LW 2m nc Fri Apr 4 06:34:18 2008 80053748
2008,80053748
TAO/15n38w/OS_15n38w_SW_2m.nc,Fri Apr 4 06:20:30 2008,55790128
ANIMATE/PAP/OS_PAP-1_200311_TSC.nc,Wed Jan 2 20:47:08 2008,582388
TAO/10s10w/OS_10s10w_SW_LW_2m.nc,Fri Apr 4 06:07:40 2008,92688128
TAO/0n140w/OS_0n140w_TVM_10m.nc,Fri Apr 4 07:09:05 2008,165957032
TAO/0n110w/OS_0n110w_TVM_10m.nc,Fri Apr 4 06:47:13 2008,149442632
10