ISIS3 I/O
Davinci, with an I/O module and an appropriately installed and configured ISIS3 installation, has a the capability to read and write ISIS3 cubes. Headers, history, and some table data support are included.
Contents |
Prerequisites
The ISIS3 module is currently in beta and not included in the build path of davinci, however it is in source control in the iomod_isis3 directory. This module has only been tested on 64-bit Linux systems (specifically CentOS 5.3 and CentOS 6.0) using ISIS 3.3 and 3.4. The 3.3 and 3.4 ISIS versions need different code. the build.sh will build the module for ISIS 3.3 and the build34.sh will build for ISIS 3.4. The resulting modules are called 'isis3.dvio' and 'isis34.dvio', respectively.
ISIS3 and QT4 (including the development headers for QT4) must be installed on your system and to compile the module, you must know where it is installed. Edit the appropriate build file in the distribution and change the values in ISIS3DIR and QTDIR to point to the proper locations. If you wish to build for 32 bit Linux systems, change the ARCH value to 32 as well. Other OSes are currently not supported, but will be in the future when the code to build it is distributed in core davinci. If you are using another operating system and/or compiler and you figure out how to compile it for it, please let us know! It will help get the build to be more portable.
Once you have edited this file to reflect the appropriate changes, Run it:
./build.sh
or
./build34.sh
If all is right, you should have a file in the directory called "isis3.dvio"
This file should be copied to where your davinci modules are kept. Alternatively, you can set the environment variable DV_MOD_PATH to the location of this file when you run davinci:
DV_MOD_PATH=directory davinci
You must have your environment set up correctly to use ISIS3. Within the Mars Space Flight Facility, simply running
$ isis3setup
will do this. Elsewhere, please consult the ISIS 3 documentation for how to do this for your environment.
When you start up davinci, you must load the IO module before it will work:
dv> insmod("isis3")
Initializing davinci/ISIS3 IO module version 0.9.0 (ISIS 3.3.x)
dv>
If you don't have your ISIS3 environment set up properly, loading the module will give you a warning:
dv> insmod("isis3")
Initializing davinci/ISIS3 IO module version 0.9.0 (ISIS 3.3.x)
Warning: ISIS Environment not setup correctly to use ISIS3 API.
Please set your ISISROOT environment variable to the root of an ISIS3 install,
or undesired operation may result.
dv>
Simple Usage: Reading a cube
When the module successfully loads, the davinci built-in function load() now knows how to read ISIS3 files without you having to take any extra action:
dv> a = load("/u/randy/isis3/I35449018RDR.cub")
Using ISIS3 API to read file '/u/randy/isis3/I35449018RDR.cub'.
1024x5308x10 array of float, bsq format [217,415,680 bytes]
dv>
Accessing Headers: load_pds
The module also supports loading the full ISIS3 structure via the load_pds function:
To read an ISIS3 cube with all headers and data, you can use the load_pds() function:
dv> a = load_pds("/u/randy/isis3/I35449018RDR.cub")
Using ISIS3 API to read file '/u/randy/isis3/I35449018RDR.cub'.
struct, 3 elements
cube: 1024x5308x10 array of float, bsq format [217,415,680 bytes]
IsisCube: struct, 9 elements
Core: struct, 7 elements...
Instrument: struct, 14 elements...
Archive: struct, 10 elements...
BandBin: struct, 5 elements...
Kernels: struct, 14 elements...
Mapping: struct, 17 elements...
AlphaCube: struct, 9 elements...
History: struct, 3 elements...
isis_struct_type: "object"
NaifKeywords: struct, 8 elements
BODY499_RADII: 3x1x1 array of double, bsq format [24 bytes]
BODY_FRAME_CODE: 10014
CLOCK_ET_-53_944976590.153_COMPUTED: "9e44da08ebb3b241"
INS-53031_TRANSX: 3x1x1 array of double, bsq format [24 bytes]
INS-53031_TRANSY: 3x1x1 array of double, bsq format [24 bytes]
INS-53031_ITRANSS: 3x1x1 array of double, bsq format [24 bytes]
INS-53031_ITRANSL: 3x1x1 array of double, bsq format [24 bytes]
isis_struct_type: "object"
Note that this form does NOT include the table data. To get the table data, you must add the suffix_data=1 option to your load_pds() call:
dv> a = load_pds("/u/randy/isis3/I35449018RDR.cub", suffix_data=1)
Using ISIS3 API to read file '/u/randy/isis3/I35449018RDR.cub'.
struct, 4 elements
cube: 1024x5308x10 array of float, bsq format [217,415,680 bytes]
IsisCube: struct, 9 elements
Core: struct, 7 elements...
Instrument: struct, 14 elements...
Archive: struct, 10 elements...
BandBin: struct, 5 elements...
Kernels: struct, 14 elements...
Mapping: struct, 17 elements...
AlphaCube: struct, 9 elements...
History: struct, 3 elements...
isis_struct_type: "object"
TableList: struct, 4 elements
InstrumentPointing: struct, 11 elements...
InstrumentPosition: struct, 10 elements...
BodyRotation: struct, 11 elements...
SunPosition: struct, 10 elements...
NaifKeywords: struct, 8 elements
BODY499_RADII: 3x1x1 array of double, bsq format [24 bytes]
BODY_FRAME_CODE: 10014
CLOCK_ET_-53_944976590.153_COMPUTED: "9e44da08ebb3b241"
INS-53031_TRANSX: 3x1x1 array of double, bsq format [24 bytes]
INS-53031_TRANSY: 3x1x1 array of double, bsq format [24 bytes]
INS-53031_ITRANSS: 3x1x1 array of double, bsq format [24 bytes]
INS-53031_ITRANSL: 3x1x1 array of double, bsq format [24 bytes]
isis_struct_type: "object"
This form now returns a structure that includes a "TableList" member, in which all the relevant table data is stored.
Writing an ISIS3 cube
To write an ISIS3 cube, you can save an array or a structure. Saving an array will create a very minimal but correct ISIS3 cube. Saving a structure requires at a minimum for the structure to have an array member called "cube."
Structures that you want to be treated as something other than an ISIS3 object should have a member in them called "isis_struct_type" that contain the values "group", "history", or "object" (which is the default). Keep in mind that groups may only contain simple values and cannot contain other structures. For more information on these objects, consult the ISIS3 PVL documentation. Normally, though, this shouldn't concern you as you have most likely started from an ISIS3 object and virtually all of the data that concerns you is already pre-populated from the load_pds call.
Example:
dv> write(a, "output.cub", type=isis3, force=1) Using ISIS3 API to write file 'output.cub'. Closing file... dv>
The write will look for the TableList member automatically, so if on the off-chance you want to write the cube back out WITHOUT the tables, you can just use the remove_struct() function:
dv> remove_struct(a, "TableList")
struct, 4 elements
InstrumentPointing: struct, 11 elements
J2000Q0: struct, 4 elements...
J2000Q1: struct, 4 elements...
J2000Q2: struct, 4 elements...
J2000Q3: struct, 4 elements...
AV1: struct, 4 elements...
AV2: struct, 4 elements...
AV3: struct, 4 elements...
ET: struct, 4 elements...
data: 8x91x1 array of double, bsq format [5,824 bytes]
cube_association: "None"
isis_struct_type: "table"
InstrumentPosition: struct, 10 elements
J2000X: struct, 4 elements...
J2000Y: struct, 4 elements...
J2000Z: struct, 4 elements...
J2000XV: struct, 4 elements...
J2000YV: struct, 4 elements...
J2000ZV: struct, 4 elements...
ET: struct, 4 elements...
data: 7x9x1 array of double, bsq format [504 bytes]
cube_association: "None"
isis_struct_type: "table"
BodyRotation: struct, 11 elements
J2000Q0: struct, 4 elements...
J2000Q1: struct, 4 elements...
J2000Q2: struct, 4 elements...
J2000Q3: struct, 4 elements...
AV1: struct, 4 elements...
AV2: struct, 4 elements...
AV3: struct, 4 elements...
ET: struct, 4 elements...
data: 8x2x1 array of double, bsq format [128 bytes]
cube_association: "None"
isis_struct_type: "table"
SunPosition: struct, 10 elements
J2000X: struct, 4 elements...
J2000Y: struct, 4 elements...
J2000Z: struct, 4 elements...
J2000XV: struct, 4 elements...
J2000YV: struct, 4 elements...
J2000ZV: struct, 4 elements...
ET: struct, 4 elements...
data: 7x2x1 array of double, bsq format [112 bytes]
cube_association: "None"
isis_struct_type: "table"
dv>
dv> write(a, "new.cub", type=isis3, force=1) Using ISIS3 API to write file 'new.cub'. Closing file... dv>
remove_struct() returns the part of the structure that was deleted, so you can store that in a variable if you want to reattach the table information again after your write:
dv> a = load_pds("/u/randy/isis3/I35449018RDR.cub", suffix_data=1)
Using ISIS3 API to read file '/u/randy/isis3/I35449018RDR.cub'.
struct, 4 elements
cube: 1024x5308x10 array of float, bsq format [217,415,680 bytes]
IsisCube: struct, 9 elements
Core: struct, 7 elements...
Instrument: struct, 14 elements...
Archive: struct, 10 elements...
BandBin: struct, 5 elements...
Kernels: struct, 14 elements...
Mapping: struct, 17 elements...
AlphaCube: struct, 9 elements...
History: struct, 3 elements...
isis_struct_type: "object"
TableList: struct, 4 elements
InstrumentPointing: struct, 11 elements...
InstrumentPosition: struct, 10 elements...
BodyRotation: struct, 11 elements...
SunPosition: struct, 10 elements...
NaifKeywords: struct, 8 elements
BODY499_RADII: 3x1x1 array of double, bsq format [24 bytes]
BODY_FRAME_CODE: 10014
CLOCK_ET_-53_944976590.153_COMPUTED: "9e44da08ebb3b241"
INS-53031_TRANSX: 3x1x1 array of double, bsq format [24 bytes]
INS-53031_TRANSY: 3x1x1 array of double, bsq format [24 bytes]
INS-53031_ITRANSS: 3x1x1 array of double, bsq format [24 bytes]
INS-53031_ITRANSL: 3x1x1 array of double, bsq format [24 bytes]
isis_struct_type: "object"
dv> temp = remove_struct(a, "TableList")
struct, 4 elements
InstrumentPointing: struct, 11 elements
J2000Q0: struct, 4 elements...
J2000Q1: struct, 4 elements...
J2000Q2: struct, 4 elements...
J2000Q3: struct, 4 elements...
AV1: struct, 4 elements...
AV2: struct, 4 elements...
AV3: struct, 4 elements...
ET: struct, 4 elements...
data: 8x91x1 array of double, bsq format [5,824 bytes]
cube_association: "None"
isis_struct_type: "table"
InstrumentPosition: struct, 10 elements
J2000X: struct, 4 elements...
J2000Y: struct, 4 elements...
J2000Z: struct, 4 elements...
J2000XV: struct, 4 elements...
J2000YV: struct, 4 elements...
J2000ZV: struct, 4 elements...
ET: struct, 4 elements...
data: 7x9x1 array of double, bsq format [504 bytes]
cube_association: "None"
isis_struct_type: "table"
BodyRotation: struct, 11 elements
J2000Q0: struct, 4 elements...
J2000Q1: struct, 4 elements...
J2000Q2: struct, 4 elements...
J2000Q3: struct, 4 elements...
AV1: struct, 4 elements...
AV2: struct, 4 elements...
AV3: struct, 4 elements...
ET: struct, 4 elements...
data: 8x2x1 array of double, bsq format [128 bytes]
cube_association: "None"
isis_struct_type: "table"
SunPosition: struct, 10 elements
J2000X: struct, 4 elements...
J2000Y: struct, 4 elements...
J2000Z: struct, 4 elements...
J2000XV: struct, 4 elements...
J2000YV: struct, 4 elements...
J2000ZV: struct, 4 elements...
ET: struct, 4 elements...
data: 7x2x1 array of double, bsq format [112 bytes]
cube_association: "None"
isis_struct_type: "table"
dv> write(a, "output.cub", type=isis3, force=1)
Using ISIS3 API to write file 'output.cub'.
Closing file...
dv> a.TableList = temp
struct, 4 elements
InstrumentPointing: struct, 11 elements
J2000Q0: struct, 4 elements...
J2000Q1: struct, 4 elements...
J2000Q2: struct, 4 elements...
J2000Q3: struct, 4 elements...
AV1: struct, 4 elements...
AV2: struct, 4 elements...
AV3: struct, 4 elements...
ET: struct, 4 elements...
data: 8x91x1 array of double, bsq format [5,824 bytes]
cube_association: "None"
isis_struct_type: "table"
InstrumentPosition: struct, 10 elements
J2000X: struct, 4 elements...
J2000Y: struct, 4 elements...
J2000Z: struct, 4 elements...
J2000XV: struct, 4 elements...
J2000YV: struct, 4 elements...
J2000ZV: struct, 4 elements...
ET: struct, 4 elements...
data: 7x9x1 array of double, bsq format [504 bytes]
cube_association: "None"
isis_struct_type: "table"
BodyRotation: struct, 11 elements
J2000Q0: struct, 4 elements...
J2000Q1: struct, 4 elements...
J2000Q2: struct, 4 elements...
J2000Q3: struct, 4 elements...
AV1: struct, 4 elements...
AV2: struct, 4 elements...
AV3: struct, 4 elements...
ET: struct, 4 elements...
data: 8x2x1 array of double, bsq format [128 bytes]
cube_association: "None"
isis_struct_type: "table"
SunPosition: struct, 10 elements
J2000X: struct, 4 elements...
J2000Y: struct, 4 elements...
J2000Z: struct, 4 elements...
J2000XV: struct, 4 elements...
J2000YV: struct, 4 elements...
J2000ZV: struct, 4 elements...
ET: struct, 4 elements...
data: 7x2x1 array of double, bsq format [112 bytes]
cube_association: "None"
isis_struct_type: "table"
dv>
History
The IO module will automatically add a history object to the ISIS3 cube, called "davisis3". This will allow us to more readily identify when an ISIS3 cube has been pushed through davinci. This does not preclude you from adding your own history object to something you want to save. For example:
dv> a = load_pds("/u/randy/isis3/I35449018RDR.cub", suffix_data=1)
Using ISIS3 API to read file '/u/randy/isis3/I35449018RDR.cub'.
struct, 4 elements
cube: 1024x5308x10 array of float, bsq format [217,415,680 bytes]
IsisCube: struct, 9 elements
Core: struct, 7 elements...
Instrument: struct, 14 elements...
Archive: struct, 10 elements...
BandBin: struct, 5 elements...
Kernels: struct, 14 elements...
Mapping: struct, 17 elements...
AlphaCube: struct, 9 elements...
History: struct, 3 elements...
isis_struct_type: "object"
TableList: struct, 4 elements
InstrumentPointing: struct, 11 elements...
InstrumentPosition: struct, 10 elements...
BodyRotation: struct, 11 elements...
SunPosition: struct, 10 elements...
NaifKeywords: struct, 8 elements
BODY499_RADII: 3x1x1 array of double, bsq format [24 bytes]
BODY_FRAME_CODE: 10014
CLOCK_ET_-53_944976590.153_COMPUTED: "9e44da08ebb3b241"
INS-53031_TRANSX: 3x1x1 array of double, bsq format [24 bytes]
INS-53031_TRANSY: 3x1x1 array of double, bsq format [24 bytes]
INS-53031_ITRANSS: 3x1x1 array of double, bsq format [24 bytes]
INS-53031_ITRANSL: 3x1x1 array of double, bsq format [24 bytes]
isis_struct_type: "object"
dv> z = struct()
struct, 0 elements
dv> z.CodeVersion = "0.1"
"0.1"
dv> z.Description = "My custom davinci process on ISIS3 cubes."
"My custom davinci process on ISIS3 cubes."
dv> a.IsisCube.History.myprocess = z
struct, 2 elements
CodeVersion: "0.1"
Description: "My custom davinci process on ISIS3 cubes."
dv> write(a, "output.cub", type=isis3, force=1)
Using ISIS3 API to write file 'output.cub'.
Closing file...
dv>
Then, upon inspecting the ISIS3 history object:
$ cathist from=output.cub
Object = spiceinit
IsisVersion = "3.3.0.3646 | 2011-09-19"
ProgramVersion = 2011-10-05
ProgramPath = /mars/common/isis3/3.3beta_64/isis/bin
ExecutionDateTime = 2011-11-01T17:16:35
HostName = Unknown
UserName = randy
Description = "Determine SPICE kernels for a camera cube"
Group = UserParameters
FROM = ./I35449018RDR.raw.cub
WEB = false
ATTACH = TRUE
CKSMITHED = FALSE
CKRECON = TRUE
CKPREDICTED = FALSE
CKNADIR = FALSE
SPKSMITHED = FALSE
SPKRECON = TRUE
SPKPREDICTED = FALSE
SHAPE = SYSTEM
STARTPAD = 0.0
ENDPAD = 0.0
URL = http://services.isis.astrogeology.usgs.gov/cgi-bin/spicein-
it.cgi
PORT = 80
End_Group
End_Object
Object = cam2map
IsisVersion = "3.3.0.3646 | 2011-09-19"
ProgramVersion = 2011-02-10
ProgramPath = /mars/common/isis3/3.3beta_64/isis/bin
ExecutionDateTime = 2011-11-01T17:16:38
HostName = Unknown
UserName = randy
Description = "Convert camera image to a map projection"
Group = UserParameters
FROM = ./I35449018RDR.raw.cub
MAP = $base/templates/maps/sinusoidal.map
TO = ./I35449018RDR.cub
MATCHMAP = false
PIXRES = CAMERA
DEFAULTRANGE = MINIMIZE
LONSEAM = AUTO
INTERP = CUBICCONVOLUTION
End_Group
End_Object
Object = myprocess
CodeVersion = 0.1
Description = "My custom davinci process on ISIS3 cubes."
End_Object
Object = davisis3
DavinciVersion = "daVinci Version #2.09"
DavinciIsis3ModuleVersion = "0.9.0 (ISIS 3.3.x)"
ExecutionDateTime = 2012-08-29T00:08:01
HostName = akagi
UserName = randy
Description = "ISIS3 cube written from davinci."
End_Object
End
$
Limitations
The table support is limited to simple homogeneous tables of the same data type. This is adequate for the NAIF/SPICE tables that are attached to spiceinit and should work for many other cases, such as "backplanes" of metadata, like latitude, longitude, solar azimuth, et cetera. ISIS3 vectors and mixed data type records are not yet supported until I can figure out a clean way to represent those within davinci data structures. This limitation allows us to use davinci arrays to store the table data, which is very efficient and meets our current use cases. This module will once again be revisited if and when use cases require more complicated table support.
