X2D Astrophotography: Shutter control app to Dither between frames

[hleon]

I'm trying to use an equatorial mount with an auto guiding auxiliary camera such as the ZWO, with the X2D as my main camera.

I'll be using the PHD2 software to control the mount and the auto guiding. In order to perform dithering (a minimal shift in the mount's position to be able to average out hot pixels and noise from each frame when stacking them in post production), I've seen many use shutter control apps such as Kstar, Sequence Generator Pro, or NINA, etc., that coordinate this mount shift between PHD2 and the camera's shutter action.

These shutter control apps work with some astronomy, DSLR, and mirrorless cameras, but I couldn't find if the X2D can be used with those apps.

Does anyone know how to dither using the X2D while autoguiding with PHD2? Or perhaps another way of autoguiding and dithering?
Your help will be much appreciated.
Hector.

[jwillson]

As far as I know, no ASCOM driver for either the X2D or any of the Fujifilm medium format cameras. The only solution I am aware of is to kick off each exposure in Phocus, then, between exposures, manually dither in PHD2 using the "Dither" button in the "manual guiding" dialog.  This will work fine, but requires you to babysit every single exposure.

- Jared

[flash]

There is NO remote shutter capability on the X2D besides the Phocus app for iPhone/iPad.

Gordon

[MGrayson]

There is NO remote shutter capability on the X2D besides the Phocus app for iPhone/iPad.

Gordon

Tethered shooting from a laptop with live view now works as well.

Matt

[grotte]

You have to establish first that there are hot pixels in your X2D sensor. Take some dark frames. I had in X1DII and found none. As for the noise, averaging over many frames is more effective than dithering. What is the focal length of you'll be using? I tried 90sec exposures with 500mm Apo-Tessar and it was about the limit without autoguiding with something like PHD2.

[jwillson]

You have to establish first that there are hot pixels in your X2D sensor. Take some dark frames. I had in X1DII and found none. As for the noise, averaging over many frames is more effective than dithering. What is the focal length of you'll be using? I tried 90sec exposures with 500mm Apo-Tessar and it was about the limit without autoguiding with something like PHD2.

All sensors have hot pixels. The factory can do things like create a bad pixel map, but it can't change the fact that thermal current is different from one pixel to the next, thus there will be fixed pattern noise that becomes visible over multiple frames. You may not notice the issue with a dark frame examined in its linear state, but if you heavily stretch it (as one naturally will do in astrophotography), then the fixed pattern noise will appear.

As to averaging over many frames being more effective than dithering... Best approach by far is to do both. On images that are well guided but un-dithered, just stacking multiple frames will address shot noise from the subject and from light pollution, but it will do nothing to address the fixed pattern noise from dark current. Dithering will move that fixed pattern noise from one frame to the next (once frames are aligned on stars), and it can then be easily removed using statistical processes such as sigma clip combine or extreme studentized deviate (or any of a dozen other common algorithms used in astrophotography). Dithering in addition to multiple exposures is a good thing. In an uncooled camera such as the X2D it is almost necessary for good results.

[hleon]

Thank you all,
I was not aware there was a manual dialog with a manual dithering ability in PHD2. I will start testing this process via Phocus with the Xcd 80 mm f1.9 and see how it works. And in post production, I'm thinking of using ASTAP, since it's free and seems to work well for others.

Will more frames at a shorter time exposure be better than less frames at longer exposure time?

Last question: in these stacking software packages, is there an algorithm such as the ones mentioned to deal with noise or hot pixels, etc for astrophotography missing making you use additional frame processing. Or do they take care of all these details within their own packages?

Any advise for a beginner in astrophotography will be greatly welcomed.
Hector.

[jwillson]

Thank you all,
I was not aware there was a manual dialog with a manual dithering ability in PHD2. I will start testing this process via Phocus with the Xcd 80 mm f1.9 and see how it works. And in post production, I'm thinking of using ASTAP, since it's free and seems to work well for others.

Will more frames at a shorter time exposure be better than less frames at longer exposure time?

Last question: in these stacking software packages, is there an algorithm such as the ones mentioned to deal with noise or hot pixels, etc for astrophotography missing making you use additional frame processing. Or do they take care of all these details within their own packages?

Any advise for a beginner in astrophotography will be greatly welcomed.
Hector.

As far as more frames at a shorter time exposure vs. fewer frames at longer exposure... There are programs that will help you calculate the "optimal" duration for a sub. It depends on a combination of factors: the focal ratio/speed of your lens/telescope, how dark your skies are, the quantum sensitivity of your sensor, and the read noise of your camera at the selected gain/ISO.  What those programs do not do is tell you the best gain/ISO to use and account for the absolutely massive amounts of storage space you can find yourself consuming with a camera such as the X2D.  The goal is to shoot long enough sub exposures that read noise become irrelevant, but short enough that you aren't losing dynamic range on stars to clipping. Here is what I would recommend:

• Take your exposures at ISO 200; Based on the "Input Referred Read Noise" graph on Photons to Photos this is where the X2D enters dual gain mode. The read noise drops substantially at this ISO. You will likely get your best overall dynamic range with this ISO.
• Shoot in RAW. Don't even think about JPG's or HEIC. You want--need--the full sixteen bits of dynamic range. The vast majority of interesting stuff in astro photos is buried in the shadows, so you need RAW images to retrieve that.
• Set white balance manually to daylight; you'll be correcting it in software later
• Take sub exposures that are long enough that the minimum values in your histogram move off the left edge of the graph. They don't need to move much--even a little away from the black point will be enough. You can use a software utility to analyze things, but most people are surprised with those utilities to find that relatively short subs will make read noise irrelevant, and you probably don't want to manage the storage and processing requirements of, say, 10s subs, so just try to expose long enough that the darkest shadows in the histogram have moved away from pure black. It is counter intuitive, but light polluted skies allow shorter subs than really dark skies. I'd start at one minute and see how that looks. The goal is not your "best looking" exposure--you'll fix that in post. The goal is just to make sure you have moved the black point off the edge of the histogram. The shot will still look way underexposed. That's normal. That's why you take lots of subs, not just one. Once you average your frames, you can stretch the heck out of them to make them look good. If you live somewhere dark, you may need to go to two or three minutes. From my backyard here in Oakland, CA, 30s is more than enough. If your subs are too short, you'll end up with a lot of data to manage. Doesn't matter too much on a 24 megapixel 12 bit camera, but it can get painful with a 100 megapixel 16 bit camera. Start at one minute and see where that gets you. As long as your sub exposures are long enough to move the histogram a bit, it makes surprisingly little difference how long they are. After processing, there is amazingly little difference between an hour captured in one minute subs, and an hour captured in ten minute subs.

Here is how the stacking software eliminates the effects of hot pixels, cosmic ray strikes, satellite trails, and decay of radioactive isotopes in your optics (yes, those are visible)...

• Some software packages will perform what is called "cosmetic correction" where the software looks for any pixels that are, say, eight or ten deviations brighter than the surrounding pixels. As long as your magnification is high enough that a single star covers multiple pixels (think telescope, not wide angle lens), these super bright pixels are necessarily defects, not stars. They can simply be removed and replaced with an average of the surrounding pixels. This is a nice feature, but won't work on wide angle shots where stars might actually be one or two pixels across. Also, not all software packages have this feature. Know it is there if you need it, but skip it for now.
• The best thing you can do is something you were already planning on--dithering. If you dither your images, then your subject moves from one frame to the next in a random way. Then, when you tell the software to "align" or "register" your frames (a necessary step whether you dither or not as no guiding is perfect), the stars will suddenly appear fixed from one frame to the next, and the fixed pattern noise will appear to move. This movement will allow the software to eliminate the noise, since it will only be in one frame in any given location. There are different algorithms that will allow you to remove the noise, each with its pro's and con's. Some work well with a small number of sub exposures but lower SNR a bit. Others require a larger number of subs, but do a better job of maintaining all signal. Read the help files of whichever software you are using to see what matches up well with your data, then experiement. The simplest algorithm is to use a "median" combine instead of an "average", but it does the most damage to SNR. Other algorithms include things like min/max (eliminating the min and max value from your stack for each individual pixel and averaging), sigma clip combining (removing any values that are a certain number of standard deviations from your average value, often in a couple passes), and it gets even fancier from there. They almost all work really well at removing fixed pattern noise from well dithered stacks of images.

General advice:

• Better (darker) skies will produce better pictures in less exposure time. Finding dark skies will do more to improve the photos than any choices in camera or technique. Just as an example, I find it takes about forty hours of exposure time from my back yard in Oakland to get the same quality result as one hour from a truly dark location.
  
• Don't assume that focus is "set it and forget it". As temperatures drop, the focal lengths on lenses change. This probably won't matter with lenses or telescopes under 120mm in focal length, but once you are talking about longer focal lengths you will need to re-focus as temps drop. Until you learn what your lens needs, plan on refocusing whenever the temp has dropped by about 1ºC.
• Often the hardest part is getting your camera/lens pointed in the right direction and focusing properly. My recommendation is to bump up the gain to something ridiculous like ISO 64,000 and then use 1s exposures to check both focus and composition. Forget autofocus. It rarely works accurately even on bright stars. If the moon is out, you can focus on that, but don't plan on using the nearby tree line--it's not far enough away. In general, unless you are photographing the Moon, try to take images when the moon isn'tout.
• It is often easier to judge focus using an iPad or iPhone rather than the rear screen or viewfinder. Mostly, this is because the camera will be pointed "up", so it is uncomfortable to look through the viewfinder. You won't do a good job if you are uncomfortable. Use Phocus.
• I don't know whether the X2D is like the X1D in terms of its implementation of electronic shutter. With the X1D, the electronic shutter mode at exposure durations greater than, I believe, 1s introduced some weird pedestal effects. Backgrounds could turn quite red/magenta with the field of view split up into four distinct quadrants with different pedestals in each. Test for this using a dark frame. Take a 10s exposure at a high ISO with the lens cap on (and your camera stuffed in a drawer to ensure there are no light leaks). Look at the image in your processing software of choice. Is there any difference between mechanical shutter and electronic? If not, use electronic shutter as it will reduce the chances of vibration. If there is a difference, use mechanical shutter. If you are adapting the camera to a non XCD lens or telescope you won't have a choice, of course, and will need to use electronic shutter. Hopefully, the X2D does not share this little problem with the X1D.
• Consider adding dark and flat frames, but not until you have had some success using just light frames. Dark signal is quite low in the Sony sensor in the X2D, so dark frames are more about fixed pattern noise and the effects of walking noise than anything else, and dithering will do more to address this than dark frames will. Flats and flat darks are nice for removing vignetting and dust motes, but, again, start with just the lights for now. Capturing good flats is surprisingly difficult.
• Blink your images before you start processing them. That is, look at each of them for at least a couple seconds at both full screen and at 100% view. Look for any images that should not be included in the stack. Did an airplane fly through the image? The stacking can do I nice job removing satellite trails, but airplane lights are just fuzzy enough along their edges that they are hard to remove. Did a passing cloud make an image much lower contrast than the rest?  Get rid of it. What about bad guiding? PHD2 is amazingly effective in general, but it won't help if you kicked the tripod or if a sudden gust of wind moved things. Look for images that are funny lookin at 100%--every star being a double exposure, funny streaks coming off stars, strange, lopsided or football shaped stars, etc. Remove these before you process. You don't need to be too ruthless, though... Just because an image is slightly less than perfect doesn't mean it can't help the final result, but obviously bad frames should be eliminated.

If I think of anything else that is good general advice for astrophotography I'll let you know. This is what comes to mind off the top of my head.

- Jared[/list]

[grotte]

Are you suggesting ISO 200 for the subs?

[hleon]

Thank you so much Jared!

[jwillson]

Are you suggesting ISO 200 for the subs?

Yes, I would suggest ISO 200 for the subs with the X2D. That is where the camera enters dual gain mode, so the read noise drops quite a bit. Full well capacity drops also, so dynamic range remains essentially unchanged from base ISO. Most people find a slightly higher gain to be easier to use than base ISO simply because it allows shorter subs with the same end result for a given amount of integration time. I would probably just standardize on ISO 200 for the X2D for astrophotography. I might shoot at more like ISO 1600 or 3200 if I weren't using a tracking mount, but for guided astrophotography (or even unguided, but with a tracking mount) I think I would stick to ISO 200.

[grotte]

Interesting. I Looked up the "Input Referred Read Noise" graph on Photons to Photos, but can't make any sense out of it: the read noise drops with ISO, but there is nothing special about the 200. The noise keeps dropping, albeit not as steeply, into the higher ISOs. What is the dual gain mode? Also there is only X1D in there, not X2D.

[hleon]

From this paper https://arxiv.org/pdf/2010.11309.pdf#:~:text=The%20pixel%20conversion%20gain%20is,the%20PPD%20to%20the%20SN. :

"The pixel conversion gain is the voltage difference that the SF creates at the column level for a single electron transferred from the PPD to the SN. Increasing this gain mitigates the impact of the noise generated at the column-level circuits which is key in low light application and also for reaching photo-electron counting capability."

Supposedly, this high level gain conversion is triggered at ISO 200 in the X2D.

The dual conversion gain refers this gain in High illumination and also in Low ilummination environments.

You might also find this article from Jim Kasson on the relation between EDR (Engineer Dynamic Range) and the ISO in the X2D:
https://blog.kasson.com/x2d/hasselblad-x2d-100c-edr-vs-iso/

Another one from Jim on the virtual Isollessness (at or after ISO 200) of the X2D: https://blog.kasson.com/x2d/x2d-isoless-visuals/

Hector.

[grotte]

Back in the day a low ISO film meant fine grain, as sparse low light photons will struggle to land on the tiny pigmented site. Increasing grain size would mean boosting ISO at expense of grainier image.  With the digital there is the quantum efficiency: how many electrons can be produced in the pixel for a given number of incident photons. (Native ISO - the intrinsic sensor characteristic). The charge, so produced, is further manipulated in great variety of ways: the current can be amplified (together with the shot noise, and other system noises), those noises can be filtered to the varying degree, and low light image enhanced in HW, again with the tradeoff being noise and digital artifacts. How significant are they is in the eye of the beholder.

If the isolessness means the image of the stack of books taken at any ISO can be made to look identical in post, I guess: ok, so? What about the image of M45? I am really looking for the practical guidance, not a scientistic discussion. If ISO200 is the sweet spot for the X1D/X2D, I'll give it a go next time I am in Joshua Tree NP.

[hleon]

Please keep us posted...