Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

4 bit data warps raster file #17

Open
feefladder opened this issue May 6, 2021 · 3 comments
Open

4 bit data warps raster file #17

feefladder opened this issue May 6, 2021 · 3 comments

Comments

@feefladder
Copy link

I have a classified raster with 4 bit data stored in a geodatabase. When converting with ARR, the output was a weird striped output. I have included the .gdb, database with a single raster. The converted file with ArcRasterRescue is lu_arr.tif, and the rest is the converted file using ArcGIS.
lu.zip
The (verbose) output is given below:


/ArcRasterRescue/build/arc_raster_rescue.exe ./lu.gdb/ 0 ./lu_arr.tif
Arc Raster Rescue (url=github.com/r-barnes/ArcRasterRescue, hash=482eb367ba86b43c, compiled=2021-05-06 20:37:05)
Opening BaseTable as './lu.gdb/a00000001.gdbtable'...
gdbtables found:
        GDB_SystemCatalog - a00000001.gdbtable
        GDB_DBTune - a00000002.gdbtable
        GDB_SpatialRefs - a00000003.gdbtable
        GDB_Items - a00000004.gdbtable
        GDB_ItemTypes - a00000005.gdbtable
        GDB_ItemRelationships - a00000006.gdbtable
        GDB_ItemRelationshipTypes - a00000007.gdbtable
        GDB_ReplicaLog - a00000008.gdbtable
        lu_prado_20m - a00000009.gdbtable
        fras_ras_lu_prado_20m - a0000000a.gdbtable
        fras_aux_lu_prado_20m - a0000000b.gdbtable
        fras_blk_lu_prado_20m - a0000000c.gdbtable
        fras_bnd_lu_prado_20m - a0000000d.gdbtable
        VAT_lu_prado_20m - a0000000e.gdbtable
Opening BaseTable as './lu.gdb/a0000000d.gdbtable'...
Opening RasterBase as './lu.gdb/a0000000d.gdbtable'
Field name (type=1): sequence_nbr
Skipping field 'sequence_nbr'
Field value: 1
Field name (type=1): raster_id
Skipping field 'raster_id'
Field value: 1
Field name (type=4): name
Field value: ''
Field name (type=1): band_flags
Skipping field 'band_flags'
Field value: 67340801
Field name (type=1): band_width
Field value: 380
Field name (type=1): band_height
Field value: 583
Field name (type=1): band_types
band_types = 0 4 20 0 00000000 00000100 00100000 00000000
Detected band data type =
Field name (type=1): block_width
Field value: 128
Field name (type=1): block_height
Field value: 128
Field name (type=3): block_origin_x
Field value: 614456
Field name (type=3): block_origin_y
Field value: 4.18933e+06
Field name (type=3): eminx
Field value: 614456
Field name (type=3): eminy
Field value: 4.17769e+06
Field name (type=3): emaxx
Field value: 622036
Field name (type=3): emaxy
Field value: 4.18933e+06
Field name (type=1): cdate
Field value: 1620332591
Field name (type=1): mdate
Field value: 1620332591
Field name (type=1): srid
Skipping field 'srid'
Field value: 2
Using geotransform (this is experimental): 614456.00000000093132257462 19.94736842105263008307 0.00000000000000000000 4189334.00000000372529029846 0.00000000000000000000 -19.96569468267581370924
Using internal datatype 'h'. Use 'c++filt -t' to decode.
Opening BaseTable as './lu.gdb/a00000009.gdbtable'...
field.raster.wkt: PROJCS["ED_1950_UTM_Zone_30N",GEOGCS["GCS_European_1950",DATUM["D_European_1950",SPHEROID["International_1924",6378388.0,297.0]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],PROJECTION["Transverse_Mercator"],PARAMETER["False_Easting",500000.0],PARAMETER["False_Northing",0.0],PARAMETER["Central_Meridian",-3.0],PARAMETER["Scale_Factor",0.9996],PARAMETER["Latitude_Of_Origin",0.0],UNIT["Meter",1.0]]
field.shape.wkt: PROJCS["ED_1950_UTM_Zone_30N",GEOGCS["GCS_European_1950",DATUM["D_European_1950",SPHEROID["International_1924",6378388.0,297.0]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],PROJECTION["Transverse_Mercator"],PARAMETER["False_Easting",500000.0],PARAMETER["False_Northing",0.0],PARAMETER["Central_Meridian",-3.0],PARAMETER["Scale_Factor",0.9996],PARAMETER["Latitude_Of_Origin",0.0],UNIT["Meter",1.0]]
Opening BaseTable as './lu.gdb/a0000000d.gdbtable'...
Opening RasterBase as './lu.gdb/a0000000d.gdbtable'
Field name (type=1): sequence_nbr
Skipping field 'sequence_nbr'
Field value: 1
Field name (type=1): raster_id
Skipping field 'raster_id'
Field value: 1
Field name (type=4): name
Field value: ''
Field name (type=1): band_flags
Skipping field 'band_flags'
Field value: 67340801
Field name (type=1): band_width
Field value: 380
Field name (type=1): band_height
Field value: 583
Field name (type=1): band_types
band_types = 0 4 20 0 00000000 00000100 00100000 00000000
Detected band data type =
Field name (type=1): block_width
Field value: 128
Field name (type=1): block_height
Field value: 128
Field name (type=3): block_origin_x
Field value: 614456.00000000093132257462
Field name (type=3): block_origin_y
Field value: 4189334.00000000372529029846
Field name (type=3): eminx
Field value: 614456.00000000093132257462
Field name (type=3): eminy
Field value: 4177694.00000000372529029846
Field name (type=3): emaxx
Field value: 622036.00000000093132257462
Field name (type=3): emaxy
Field value: 4189334.00000000372529029846
Field name (type=1): cdate
Field value: 1620332591
Field name (type=1): mdate
Field value: 1620332591
Field name (type=1): srid
Skipping field 'srid'
Field value: 2
Using geotransform (this is experimental): 614456.00000000093132257462 19.94736842105263008307 0.00000000000000000000 4189334.00000000372529029846 0.00000000000000000000 -19.96569468267581370924
Opening BaseTable as './lu.gdb/a0000000a.gdbtable'...
Opening RasterProjection as './lu.gdb/a0000000a.gdbtable'
Opening BaseTable as './lu.gdb/a0000000c.gdbtable'...
Allocating 1x384x640 = 245760
Opening Raster Data as ./lu.gdb/a0000000c.gdbtable
Skipped points: 0
Projection: PROJCS["ED_1950_UTM_Zone_30N",GEOGCS["GCS_European_1950",DATUM["D_European_1950",SPHEROID["International_1924",6378388.0,297.0]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],PROJECTION["Transverse_Mercator"],PARAMETER["False_Easting",500000.0],PARAMETER["False_Northing",0.0],PARAMETER["Central_Meridian",-3.0],PARAMETER["Scale_Factor",0.9996],PARAMETER["Latitude_Of_Origin",0.0],UNIT["Meter",1.0]]

I don't think it is related to the issue, but I also had some overflow warnings when building the program related to the nodata value.

```

Scanning dependencies of target arc_raster_rescue
make[2]: Warning: File 'CMakeFiles/arc_raster_rescue.dir/depend.make' has modification time 0.71 s in the future
[ 25%] Building CXX object CMakeFiles/arc_raster_rescue.dir/src/arr.cpp.o
/mnt/c/Users/Joep/Documents/ArcRasterRescue/src/arr.cpp: In instantiation of ‘RasterData::RasterData(std::string, const RasterBase&) [with T = unsigned char; std::string = std::__cxx11::basic_string]’:
/mnt/c/Users/Joep/Documents/ArcRasterRescue/src/arr.cpp:1246:20: required from ‘void ExportTypedRasterToGeoTIFF(std::string, std::string, int, std::string) [with T = unsigned char; std::string = std::__cxx11::basic_string]’
/mnt/c/Users/Joep/Documents/ArcRasterRescue/src/arr.cpp:1278:84: required from here
/mnt/c/Users/Joep/Documents/ArcRasterRescue/src/arr.cpp:958:36: warning: unsigned conversion from ‘int’ to ‘unsigned char’ changes value from ‘-9999’ to ‘241’ [-Woverflow]
958 | resize(maxpx-minpx, maxpy-minpy, -9999);
| ^~~~~
/mnt/c/Users/Joep/Documents/ArcRasterRescue/src/arr.cpp:1132:13: warning: unsigned conversion from ‘int’ to ‘unsigned char’ changes value from ‘-9999’ to ‘241’ [-Woverflow]
1132 | no_data = -9999; //TODO: This cannot always be NoData.
| ^~~~~
/mnt/c/Users/Joep/Documents/ArcRasterRescue/src/arr.cpp: In instantiation of ‘RasterData::RasterData(std::string, const RasterBase&) [with T = signed char; std::string = std::__cxx11::basic_string]’:
/mnt/c/Users/Joep/Documents/ArcRasterRescue/src/arr.cpp:1246:20: required from ‘void ExportTypedRasterToGeoTIFF(std::string, std::string, int, std::string) [with T = signed char; std::string = std::__cxx11::basic_string]’
/mnt/c/Users/Joep/Documents/ArcRasterRescue/src/arr.cpp:1284:83: required from here
/mnt/c/Users/Joep/Documents/ArcRasterRescue/src/arr.cpp:958:36: warning: overflow in conversion from ‘int’ to ‘signed char’ changes value from ‘-9999’ to ‘-15’ [-Woverflow]
958 | resize(maxpx-minpx, maxpy-minpy, -9999);
| ^~~~~
/mnt/c/Users/Joep/Documents/ArcRasterRescue/src/arr.cpp:1132:13: warning: overflow in conversion from ‘int’ to ‘signed char’ changes value from ‘-9999’ to ‘-15’ [-Woverflow]
1132 | no_data = -9999; //TODO: This cannot always be NoData.
| ^~~~~
[ 50%] Linking CXX static library libarc_raster_rescue.a
make[2]: warning: Clock skew detected. Your build may be incomplete.
[ 50%] Built target arc_raster_rescue
Scanning dependencies of target arc_raster_rescue.exe
make[2]: Warning: File 'CMakeFiles/arc_raster_rescue.exe.dir/depend.make' has modification time 0.7 s in the future
[ 75%] Building CXX object CMakeFiles/arc_raster_rescue.exe.dir/evaluation/main.cpp.o
[100%] Linking CXX executable arc_raster_rescue.exe
make[2]: warning: Clock skew detected. Your build may be incomplete.
[100%] Built target arc_raster_rescue.exe

</details>

I hope this helps!
@r-barnes
Copy link
Owner

I've moved the no data issue to #18: please try to keep issues separate where possible.

I think somewhere I have an 8-bit assumption hard-coded. If you had the exact same dataset saved to, e.g., 4-bit, 8-bit, 16-bit, and 32-bit integers it might be possible to figure out how Arc is indicating a 4-bit encoding.

@feefladder
Copy link
Author

feefladder commented May 12, 2021

I'll see if I can get that data! Would it be four .gdb files with those datasets or one with four datasets?

As for splitting the issues, I was not sure if it was rrlated, so I didn't want to make an issue just yet. Thanks for making it!

@feefladder
Copy link
Author

So I've added the .gdb files:
LUs.zip
it's a bit funny though: In the properties tab, the data is given as char for 8-bit. (short for 16 bit and signed int for 32 bit are normal I think.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants