Refactor image/stamp workflow #424

g-braeunlich · 2023-09-28T12:13:48Z

No description provided.

welucas2 · 2024-02-22T13:55:55Z

The latest commit aligns photon objects in the photon pooling pipeline with their positions in the original pipeline.

welucas2 · 2024-04-29T14:43:27Z

Almost there, but blocked by GalSim-developers/GalSim#1284.

… be made if the full image has an even number of pixels in x and/or y, then apply this offset to photon arrays before merging.

…age.

…from stamp.py.

welucas2 · 2024-06-05T13:35:15Z

This is finally open for review -- all comments welcome.

rmjarvis

Thanks William. I have a bunch of comments, but this is definitely a good start. If it would be helpful to talk through any of this on a zoom call, let me know.

rmjarvis · 2024-08-14T17:47:44Z

config/imsim-config.yaml

+    # If using photon pooling, FFT and photon objects are batched separately.
+    # There will probably be few enough FFT objects to do in a single batch.
+    # Bright photon objects are treated in all photon batches, but at 1/nbatch_photon of
+    # their flux. Faint photonobjects are placed at their full flux in random batches.


Suggested change

# their flux. Faint photonobjects are placed at their full flux in random batches.

# their flux. Faint photon objects are placed at their full flux in random batches.

Thanks for spotting this. Fixing in the next commit.

rmjarvis · 2024-08-14T17:49:56Z

config/imsim-config.yaml

+    # These parameters only affect LSST_PhotonPoolingImage images.
+    # The batch numbers can be changed if desired.
+    nbatch_fft: 1
+    nbatch_photon: 10


I think this should be the same parameter as the simple nbatch. We can just document that with photon pooling, nbatch refers only to the photon-shooting objects. Then nbatch_fft can be an optional parameter that defaults to 1. (Maybe not even worth mentioning here if most users wouldn't change it?)

I agree with this. Nice to keep the number of parameters down. nbatch_photon will be no more, and the photon pooling will be controlled directly with nbatch.

rmjarvis · 2024-08-14T17:53:58Z

imsim/lsst_image.py

+
+class LSST_ImageBuilder(LSST_ImageBuilderBase):
+    """This is mostly the same as the GalSim "Scattered" image type.
+    So far the only change is in the sky background image."""


This doc is probably obsolete. There are quite a few differences now.

Happy to change this. Can you suggest something more suitable?

rmjarvis · 2024-08-14T17:55:18Z

imsim/lsst_image.py

-                'x' : { 'type' : 'Random' , 'min' : xmin , 'max' : xmax },
-                'y' : { 'type' : 'Random' , 'min' : ymin , 'max' : ymax }
-            }
+        set_config_image_pos(config, base)


Rather than making this a free function in a different file, this should probably be a private method of the base class, which both subclasses can call.

Agreed, doing this.

rmjarvis · 2024-08-14T18:01:06Z

imsim/lsst_image.py

+            base["stamp"]["photon_ops"] = []
+        photon_ops = base["stamp"]["photon_ops"]
+        shift_op = {'type': 'Shift'}
+        shift_index = next((index for (index, d) in enumerate(photon_ops) if d["type"] == "Shift"), None)


I don't love this implementation. I get why something like this is now required. But I think I'd rather have RubinOptics and RubinDiffractionOptics just have an optional parameter to apply this shift or not. Then when running this class, base[image_pos] is defined and they can apply the shift based on that.

In the photon pooling class, once we've pooled all the photons, we should set the object-specific values (image_pos, sky_pos, maybe others) that apply to a particular object to None. Then the RubinOptics and RubinDiffractionOptics photon ops would see that the image_pos isn't defined, so not apply any shift.

This feels to me more elegant and possibly more efficient.

This should now be done -- see RubinOptics and RubinDIffractionOptics. If an optional shift_photons = True is passed, the shift is performed.

rmjarvis · 2024-08-16T21:24:16Z

tests/test_image.py

+        },
+        "psf": {"type": "Convolve", "items": [{"type": "Gaussian", "fwhm": 0.3}]},
+        "stamp": {
+            "type": "LSST_Silicon",


This should be the new stamp type when image_type is LSST_PhotonPoolingImage

Yes, photon pooling now needs to use the new stamp type rather messing with base. The test sets the correct stamp type.

rmjarvis · 2024-08-16T21:29:59Z

tests/test_image.py

+
+def test_lsst_image_photon_pooling_pipeline():
+    """Check that LSST_PhotonPoolingImage batches objects as expected and renders objects at the correct positions."""
+    run_lsst_image("LSST_PhotonPoolingImage")


I feel like we probably want a few additional tests in this pipeline. E.g. that the FFT and faint objects only got drawn once. And the others got drawn nbatch times.

I've written a set of tests for the partitioning and batching process. As mentioned above, it also confirms that total flux is conserved for photon objects,

rmjarvis · 2024-08-16T21:33:36Z

tests/test_image.py

+        help="Similar to -k of pytest / unittest: restrict tests to tests starting with the specified prefix.",
+        default="test_",
+    )
+    args = parser.parse_args()


FWIW, when working on stuff, I usually just put the name of the test function I'm working on at the start of this name == '__main__' block, followed by quit(). Then remove it before committing. Seems easier than rolling all of this argparse stuff, but 🤷.

This is also done in test_diffraction_fft.py, but I'm happy to change it if you'd like me to.

rmjarvis · 2024-08-16T21:37:42Z

tests/test_photon_ops.py

-    expected_x_pic_center = 564.5
-    expected_y_pic_center = -1431.4
+    expected_x_pic_center = -989.5971378245167
+    expected_y_pic_center = -3840.3512012842157


Why are these (and subsequent tests) changing so much? This isn't close.

Regression tests are supposed to make sure we don't change the functionality of existing code. I know the implementation of these photon ops changed, but I think we want the tests to remain the same or at least very very close. What happened here?

This is still a big issue. I'm 99% convinced the error is in the original test itself. My current best guess is that with the old code it may not have been internally self-consistent around the WCS used, while the new XyToV should enforce self-consistency. But it still needs to be looked at.

rmjarvis · 2024-08-16T21:39:14Z

tests/test_stamp.py

@@ -52,7 +53,8 @@ def create_test_config():
            }],
        },
        "bandpass": galsim.Bandpass('LSST_r.dat', wave_type='nm'),
-        "wcs": galsim.PixelScale(0.2),
+        "wcs": wcs,
+        "current_image": galsim.Image(1024, 1024, wcs=wcs)


Why was this change required?

The change to the wcs field isn't actually necessary, it looks. The addition of current_image is needed for deserialize_rubin_optics, deserialize_rubin_diffraction_optics and deserialize_rubin_diffraction which uses it to get img_wcs, required by the new version of XyToV.

…from LSST_SiliconBuilder.

… creating a new builder on the spot.

…e a single consolidated stamp type LSST_Photons for all types of objects.

…re no longer cached, leaving the index, phot_flux and processing mode only.

…mp is being used.

…ine. Photon pooling translates relative to full_image.center just before accumulation instead.

welucas2 · 2024-11-07T13:44:11Z

The latest set of commits should address most of the comments above. I'll respond to those individually. A couple do still need to be looked at.

rmjarvis · 2024-11-19T19:46:04Z

Are you waiting on me for anything here? I thought you still had some more comments to address, so I was waiting for that to be done. But if it would be helpful for me to review this again at this point, let me know.

welucas2 · 2024-11-20T16:00:17Z

Not just yet -- I have a couple more commits on the way with fixes and tests for the batching. I'll push these later today or tomorrow, and then I think that should be everything ready for review again.

…he functions there its static methods.

…w type.

…beginning. Previously missed. Ensure conservation of photon object fluxes during batching by adding the modulo to random batches.

…ST_ImageBuilderBase.

…than the (0,0) position.

welucas2 · 2024-11-20T17:59:32Z

I think this is ready for re-review now, though CI is failing with an error in test_stamp_bandpass_airmass from numpy coming through GalSim. I saw the same error a couple of weeks ago on an ARM system when using GalSim 2.6 which introduces its own utilities.least_squares(). I wasn't able to investigate further before the system went down and only came back up yesterday.

welucas2 · 2024-11-21T18:52:42Z

I think there is actually one more commit incoming. I've realised that in edge cases where nominally bright photon objects' phot_flux falls below nbatch, we can end up with batches containing objects with 0 flux. I think I have a fix, by having these objects be treated as faint objects for batching purposes. I'll test this and if all is well push it tomorrow.

…faint objects.

…ther photon pooling tests.

welucas2 · 2024-11-22T17:05:29Z

OK, that should be it -- code open for review!

g-braeunlich requested a review from rmjarvis September 28, 2023 12:13

g-braeunlich self-assigned this Sep 28, 2023

g-braeunlich force-pushed the u/g-braeunlich/refactor-image branch 2 times, most recently from e58012d to c145f0c Compare October 10, 2023 12:34

g-braeunlich force-pushed the u/g-braeunlich/refactor-image branch from c145f0c to 1d7fefc Compare October 30, 2023 09:16

g-braeunlich force-pushed the u/g-braeunlich/refactor-image branch 2 times, most recently from acbac83 to 9f49df7 Compare November 30, 2023 12:45

welucas2 self-assigned this Apr 29, 2024

welucas2 mentioned this pull request Apr 30, 2024

Save photons from the component objects of a ChromaticSum GalSim-developers/GalSim#1285

Closed

welucas2 force-pushed the u/g-braeunlich/refactor-image branch from daae77c to 7290a11 Compare May 24, 2024 12:11

g-braeunlich and others added 14 commits June 5, 2024 14:09

Refactor image/stamp workflow

62e8a7b

Add more doc for the photon pooling code

85cb394

Log photon objects and fix faint object flux

4920810

Fix test_image.py photon object list

e386d09

Saner batch sizes and photon pooling checkpoints to separate dataset

84115b2

Move photon pooling functions into photon_pooling.py and add docstrings.

8ead410

Automatically add Shift photon operator for LSST_Image runs.

b93f3d1

Determine each photon object's sub-pixel offset and the adjustment to…

d789e81

… be made if the full image has an even number of pixels in x and/or y, then apply this offset to photon arrays before merging.

Use copyFrom() rather than assignAt() to merge photon arrays.

d474e34

Ensure Shift photon op is used automatically with RubinOptics.

17de2ad

Fix rebase and separate tests for LSST_Image and LSST_PhotonPoolingIm…

ba11bcb

…age.

Correct use of nbatch_per_checkpoint for LSST_Image.

0413770

Re-add Shift operator manually and correct photon shooting decision.

d481c01

Correct auto-insertion of Shift photon operator and remove build_gal …

b786cb0

…from stamp.py.

welucas2 force-pushed the u/g-braeunlich/refactor-image branch from bdaadae to b786cb0 Compare June 5, 2024 13:10

welucas2 marked this pull request as ready for review June 5, 2024 13:32

Merge branch 'main' into u/g-braeunlich/refactor-image

0948e3c

rmjarvis requested changes Aug 16, 2024

View reviewed changes

welucas2 added 8 commits October 15, 2024 15:14

Fix typo in imsim-config.yaml and remove nbatch_photons.

0a2ab42

Rename PhotonStampBuilder to LSST_PhotonsBuilder and have it inherit …

b26b77b

…from LSST_SiliconBuilder.

build_obj() gets the stamp builder from valid_stamp_types rather than…

cc1686e

… creating a new builder on the spot.

Photon pooling with LSST_PhotonPoolingImage now has build_stamps() us…

c751dbe

…e a single consolidated stamp type LSST_Photons for all types of objects.

StellarObject becomes ObjectCache. object, PSF, offset and fft_flux a…

98f1024

…re no longer cached, leaving the index, phot_flux and processing mode only.

Create new config template for use in photon pooling runs.

587fc48

LSST_PhotonPoolingImage checks during setup that the LSST_Photons sta…

a45300b

…mp is being used.

Remove Shift photon op. Applied within RubinOptics for original pipel…

c5c553a

…ine. Photon pooling translates relative to full_image.center just before accumulation instead.

welucas2 added 6 commits November 20, 2024 16:54

Move LSST_PhotonPoolingImageBuilder into photon_pooling.py and make t…

d897327

…he functions there its static methods.

Use simple seq of DeltaFunctions in test_image config, rather than ne…

9f1e73b

…w type.

Rename StellarObject in comments and stellar_obj variable

430240b

make_batches adds remainder objects to early batches starting at the …

c3b4737

…beginning. Previously missed. Ensure conservation of photon object fluxes during batching by adding the modulo to random batches.

Add partitioning and batching tests for photon pooling.

268e1f7

Make create_full_image and set_config_image_pos private members of LS…

eeb67dc

…ST_ImageBuilderBase.

welucas2 force-pushed the u/g-braeunlich/refactor-image branch from 46461fb to eeb67dc Compare November 20, 2024 16:54

PhotonPooling creates a local WCS with full_image.true_center rather …

d1fe74d

…than the (0,0) position.

welucas2 added 2 commits November 22, 2024 17:02

Batch photon objects with lower flux than nbatch as though they were …

8f0d002

…faint objects.

Add test for partitioning of low flux photon objects and strengthen o…

6bc019f

…ther photon pooling tests.

welucas2 requested a review from rmjarvis November 22, 2024 17:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor image/stamp workflow #424

Refactor image/stamp workflow #424

g-braeunlich commented Sep 28, 2023

welucas2 commented Feb 22, 2024

welucas2 commented Apr 29, 2024

welucas2 commented Jun 5, 2024

rmjarvis left a comment

rmjarvis Aug 14, 2024

welucas2 Aug 21, 2024

rmjarvis Aug 14, 2024

welucas2 Aug 21, 2024

rmjarvis Aug 14, 2024

welucas2 Nov 20, 2024

rmjarvis Aug 14, 2024

welucas2 Nov 20, 2024

rmjarvis Aug 14, 2024

welucas2 Nov 20, 2024

rmjarvis Aug 16, 2024

welucas2 Nov 20, 2024

rmjarvis Aug 16, 2024

welucas2 Nov 20, 2024

rmjarvis Aug 16, 2024

welucas2 Nov 20, 2024

rmjarvis Aug 16, 2024

welucas2 Nov 20, 2024

rmjarvis Aug 16, 2024

welucas2 Nov 20, 2024

welucas2 commented Nov 7, 2024

rmjarvis commented Nov 19, 2024

welucas2 commented Nov 20, 2024

welucas2 commented Nov 20, 2024

welucas2 commented Nov 21, 2024

welucas2 commented Nov 22, 2024

	# their flux. Faint photonobjects are placed at their full flux in random batches.
	# their flux. Faint photon objects are placed at their full flux in random batches.

Refactor image/stamp workflow #424

Are you sure you want to change the base?

Refactor image/stamp workflow #424

Conversation

g-braeunlich commented Sep 28, 2023

welucas2 commented Feb 22, 2024

welucas2 commented Apr 29, 2024

welucas2 commented Jun 5, 2024

rmjarvis left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

welucas2 commented Nov 7, 2024

rmjarvis commented Nov 19, 2024

welucas2 commented Nov 20, 2024

welucas2 commented Nov 20, 2024

welucas2 commented Nov 21, 2024

welucas2 commented Nov 22, 2024