Blog

Compressing FITs files

In the days of consumer high capacity storage things look great but modern CMOS cameras output fairly large sized FITs but coupled with short exposure sub one evening can result in a large amount of data to process.

So various solutions are available – block compression and byte compression. Taking one FITs image (test.fit) from a OSC CMOS we can compare different compression methods.

Windows Compression

Using the built compression accessed via a file’s advanced options the file is reduced from 49.8MB down to a 43.5MB of disk usage.

Gzip compression

Using the Linux gzip compression utility (v1.9) we perform a byte-level compression in userspace.

# cksum test.fit
4076709869 52223040 test.fit
# gzip -9 test.fit
# ls -lashi test.fit.gz
524306 33M -rw-r--r--. 1 root root 33M Jul 15 14:34 test.fit.gz
# du -sh test.fit.gz
33M     test.fit.gz
# cksum test.fit.gz
1554386539 34046021 test.fit.gz
# gunzip test.fit.gz
# cksum test.fit
4076709869 52223040 test.fit

Gzipping the 50MB fits files results in 33MB of disk usage. Using fpack (v1.7.0) results in similar results and the file format (fit.fz). The fit.fz file is supported by the processing tool Pixinsight.

# fpack -g -q0 -v test.fit

Block Compression

Here we utilise the block-level compression feature of IBM Spectrum Scale Developer Edition.

# ls -lashi test.fit
524305 50M -rw-r--r--. 1 root root 50M Jul 15 14:34 test.fit
# du -sh test.fit
50M     test.fit
# cksum test.fit
4076709869 52223040 test.fit
# /usr/lpp/mmfs/bin/mmchattr --compression z test.fit
# ls -lashi test.fit
524305 36M -rw-r--r--. 1 root root 50M Jul 15 14:34 test.fit
# cksum test.fit
4076709869 52223040 test.fit
# /usr/lpp/mmfs/bin/mmchattr --compression no test.fit
# du -sh test.fit
36M     test.fit

The compression is transparent to the user application and user and they seen as the original file, modified files will need to be recompressed. As this is at the file system block-level the checksum is the same if the file is compressed or not.

Pixinsight XIF Compression

Pixinsight offers the XISF format as an alternative to the FITs format. Existing FITs file can be converted to XISF using the BatchFormatConversion script. To utilise compression you need to supply the appropriate output hints.

Pixinsight Batch Format Conversion

The process console output shows the resulting size of the converted FITs image being reduced to 49% of its original size.

Pixinsight Process Console

Conclusion

Pixinsight XIF compression gives the best compression but the issue is that it is currently not currently an accepted standard and other utilities (astropy) do not have the ability to read them. Pixinsight can also read fit.fz files

NINA can save camera files as FITS or XISF, the XISF method offers LZ4, LZ4HC and Zlib compression but no gzip/fpack option for FITS.

For now I will continue to compress FITS to fit.gz or fit.fz to give me the benefits of space savings whilst still allowing me to use the data across Windows and Linux utilities.

Viewing Report Thursday 30th June 2022 Tenerife – Parador

21:30 – 06:30

Observing setup

First of all tonight I have taken darks for 300s that are needed for the earlier data. However I am still suffering from the QHY168c disconnecting from the laptop, this time in TSX. I have upgraded the software yesterday but this has not made a difference, so it must be something to do with the way there USB cable is connecting. I will attempt to troubleshoot when I get home. Right now I have switched to Ezcap again which seems to be ok.

I have setup over by Mark and will see what this location is like behind the pool building. I have found that the WiFi is weak or non existent. Also the staff room is nearby and the light is constantly on so maybe not the best place to image from.

@00:03 NGC 6352 30s -20℃ I took a single exposure of this.

@00:19 LDN 10 30s -20℃

LDN 10

@00:42 Haumea 30s -20℃

Haumea

@01:26 Makemake 30s -20℃

Makemake

@01:52 Pluto 30s -20℃

Pluto

@02:33 Eagle nebula 30s -20℃

M16 Eagle Nebula

@03:45 NGC 6520 &| B86 30s -20℃

NGC 6520 & B86 ‘Ink Spot’

Viewing Report Tuesday 28th June 2022 Tenerife – Parador

22:05 – 05:50

Pond at the Parador

@22:30 Omega Centauri 30s -20℃

Omega Centauri

@23:17 Centaurus A 30s -20℃

Centaurus A

@00:02 M24 30s -20℃

@01:33 M6 30s -20℃

@02:38 M25 30s -20℃

@03:26 LDN 564 300s -20℃

Flats taken

Darks 300s -20℃ (take tomorrow) / 0.35s -25℃ – DONE

Viewing Report Monday 27th June 2022 Tenerife – Parador

22:00 – 07:00

Skywatcher Esprit 120 ED

First thing tonight is to complete taking the darks from yesterday, so I need 20 x 120s -20℃, but also the flat darks I forgot for the 0.35ms flats. Then I will move to taking longer images tonight. The challenge with open clusters is that you need short exposures so you don’t oversaturate the stars.

@22:50 Omega Centauri 30s -20℃

Omega Centauri

@23:20 Centaurus A 300s -20℃

Centaurus A

@00:09 Makemake 120s -20℃ x 10

@00:29 Haumea 120s -20℃ x 10

@01:07 M8 Lagoon 300s -20℃ x 24 (focus drifted so review later images)

M8 Lagoon

@03:22 Comet C/2017 K2 PANSTARRS 120s -20℃ x 7

@03:53 M21 30s -20℃ x 90

@05:00 M20 & M21 120s x5

M20 & M21

@05:45 LDN 574 300s x 9 which was a mistake

LDN 574

Dark Flats – 0.35s -20℃ – DONE / 0.35s – 25℃ (tomorrow)

Flats – 0.35s -20℃ – DONE

Darks 30s -20℃ – DONE / 300 -20℃ (tomorrow) / 120 -20℃ DONE

Viewing Report Sunday 26th June 2022 Tenerife – Parador

22:55 – 08:00

Teide

Finally got our bags today from the airport. Having arrives on the 2:05pm flight and not hearing from BA by 4pm, Lawrence and I headed off down the mountain in search of the 4 missing bags. It took just over an hour to arrive back in the Cicar car park.

Once there we headed inside to be told the the Iberia desk, which deals with lost and late baggage was only accessible the arrivals hall and so we were told to go through the exit doors and not to worry about the police. So we nervously😱 walked through and were greated with the sight of all the lost bags!

Blurry Bags

It took some 20 mins to sort through the paperwork before we were pushing the bags back to the car for the 1 hour drive back up the TF1 and TF38.

After a hearty dinner, which was absolutely delicious and filling, we headed outside around 9pm to start setting up. This took me the best part of 2 hours to put the gear together after ferrying it from upstairs and then another 60 mins to calibrate and be ready to image. First up tonight is M23.

@ 23:55 started on M23 Open Cluster, 30s -25℃ exposures so not to saturate the stars.

M23

after 87 images escape hung!

@ 01:20 started on M18 Open Cluster 60s, 10 -20℃ or so images this time using TSX to image and then refocused and took 80 images in total.

M18

@ 3:22 started on M26 Open Cluster -20℃ took 68 at 60s

Pluto 4:41am 120s -20℃ I took 10 frames

Pluto

@5:21 M73 Open Cluster 60s -20℃ took 60 frames

M73

Flats 0.35s -20 & 0.35s -25℃

Darks – 30s -25℃ – Done / 60s -20℃ – Done / 120s -20℃ (tomorrow night) / Flat Darks (tomorrow night)

Viewing Report 1st June 2022

22:40 – 00:31

Visual this evening on the 22″ Dob. The 120 Esprit is still off the MEII so the IMT3b observatory is out of action for a few weeks whilst it is not balanced. The MyT and 120 Esprit are now packed away for the Tenerife trip which is really exciting, first time in over 2 years!

Bob came round this evening and we explored the rather bright sky given the time of year with the dob. So what what did we see?

Observing Notes

M5 seen 23:21 31mm and 13mm better than M3

M3 resolved easily 23:17 31mm and  13mm more compact than M5

M51 massive filled fov in 13mm much smaller in 31mm

M92 seen both eyepieces and very bright

M57 13mm only very large AV central star, very big ring

Viewing Report 27th May 2022

19:48 – 03:08

Mobile rig

Tonight is the first night of testing the mobile rig for our 2022 Tenerife BASEG trip. I have just started to setup and am waiting for night to fall. The weather is warm and clear and I am on the IMT3b observing patio.

Tonight is about making sure I can take a set of images and guide and that everything works.

First thing to note is on firing out TSX that I need to change the settings for the mount and select usbmodem1442101 and then connect. This did not work the first time and resulted in an error, disconnecting the USB from the 2 port hub and reconnecting fixed the problem.

Ready to image

Took 44 x 300s on M101 and started to pack up at 3:08 when it was getting light which I could see on the SQM.

The next morning zzzzzz
M101 300s Exposure

Viewing Report 25th April 2022

22:01 – 04:23

M96 galaxy in LRGB on the 12″. Focus position 20691. Started at 22:29. Set up for 4 hours before it disappears behind the trees on the West side of the Meridian. Looking to take 24 x Lum and 8 x RGB all at 300s. Left the dome running and off to bed.

M96 Luminance 300s

The next morning I reviewed the status of the dome and imaging run and it ran al night without a glitch which was great. However the focus drifted quite considerably so I will need to throw away some of the data. What I need to do is cool down the observatory and the mirror before hand which last night I did not due to the last minute decision to image. Also getting the temperature compensation working on the focuser would help, so maybe time to revisit.

I will do another run on M96 on the next clear night.

Exoplanet TOI-1789b (15/4/2022)

19:35 – 00:40

Tonight is a synchronous group observing campaign with Exoclock. We are going after an exoplanet around star Tycho 1962:0303 / HD 82139. I note that HD 82139 does not come up in TSX but the Tycho star in the right format does. I tried to install the 6GB USNO B database but it fails with an error that the files are not all in the directory. I will try to sort this out later.

Start of transit was 22:28 BST Mid transit 23:37 BST , End of Transit is 00:47 BST, 1 hour after transit is 01:47 BST

https://www.exoclock.space/database/planets/TOI-1789b/

I had an initial problem after starting the imaging run, the focuser went back to 14000 again which meant I was out of focus. I only noticed later and then restarted, fortunately before the start of the transit.

Reviewing the files the following evening I noted the dome closed around 00:40 so 7 mins before the end of Transit. I have no idea why the dome closed…….

Transit submitted

Viewing Report 27th March 2022

19:28 – 04:35

M36 with Esprit 120 from the dome until 23:45. I took 60 x 60s and 60 x 120s to try and reduce star satuaration.

M36 60s exposure

To guide on the 12″ tonight I realised the camera was not in focus, so I connected the focuser and moved from pos 14000 to pos 21000 which allowed me to get a much higher than 25-30 star signal to noise ratio SNR than before when it was sub 10 and this guided perfectly with perfect star profiles too.

I then went to bed, awoke early at 1am and set the Esprit imaging M59 and took 36 x 300s before heading back to bed. The imaging stopped at 4:35am. Later in the morning when I awoke I took Flats as I had darks from the other night.

M59 300s exposure

Viewing Report 26th March 2022

19:47 – 03:00

Opened the dome and setup the 22″ Obsessions telescope for visual.

I have gone with a recommendation of Mark Radice this evening andI will both image and perform visual on M46 including the planetary nebula Herschel H39-4 that resides within it.

So with the 120 Esprit inside the dome imaging away on M46, now that it is to the East of the Meridian, I connected the Argo Navis to the 22″ and set about aligning the scope. After a few minutes and with the alignment complete, I pushed to M46.

M46 Observing Notes

31mm Nagler 75x Mag. M46 Open Cluster fills the view. Apparent instantly is the planetary nebula Herschel H39-4 towards the 5 o’clock position. A small ring can be seen with direct vision and with averted vision the contrast increases. I am not using any filter yet.I can resolve many stars. With the 13mm Ethos 180 x Mag H39-4 becomes much larger and you can resolve easily the star within the centre of the ring. Now only several handfuls of stars can be resolved.

M42 Observing Notes

31mm Nagler 75 x Mag. M42 steller nursary and hydrogen nebula is very bright with the wings sweeping outward. The trapezium is clearly seen. With averted vision much more gas is forthcoming around the area within the FoV. Moving to the 13 Ethos 180 x Mag the trapezium is resolved pleasantly into a much wider set of 4 stars. The hydrogen gas now takes on structure and lingering on this one can make out dark features within it give the nebula a 3 dimensional feel. Now you experience a warm glow within the eyepiece that seems to draw your eye towards the gas in which the trapezium stars sit.

At this point I cam back into there Warmroom to write up my notes and review the images being taken in the observatory. I have now taken 21 x 300s of M46. Now switching to M48.

I then joined the BAS Zoom call to speak with Derek, Nigel, Bob and Mil Dave. We discussed M44 taken by Bob and suggested if I could see the UGC 4526 galaxy. So I went out to take a look.

M44 and UGC 4526 from Bob

M44 Observing Notes

Looking at M44 with Nagler 31mm at 75x Mag…..

I landed up finishing M48 at 23:47. I took 30 x 300s but then checked through the last few when the dome closed and I actually got 26, I might have lost the guide or the dome slewed incorrectly or shut, not sure which.

Addendum – It turned out to be loosing the guide star and because the camera wears not in focus, so connecting the focuser and moving from 14000 to 21000 position fixed it. The star profile was much better and SNR was 25-30 instead of 10.

Viewing Report 23rd March 2022

19:14 – 21:43

Another clear night, so many in a row and a lovely warm spring day of 18℃ I’m out again 🙂

I’m grabbing some more M61 luminance tonight before I move onto RGB. Some teething problems made me start much later than desired, including a cable coming out the back of the mount for the power to the QHY168C, not sure why. So I did not get going until around 8:30pm.

I managed to grab 21 luminance frames at 300s until around 11:30pm when the the guider lost it’s stars. This is a setting problem as it was clear all night. I knew before I went to bed that there was an issue as I had seen the guider stop guiding. I will take a look at it and calibrate the guider using guider assistant after BAS tomorrow night.

Viewing Report 21st March 2022

19:00 – 21:17

A quick imaging run this evening, again for NGC 1999, this time LRGB, 5 frames of each at 300s then it was too low and my bed time on a school night 😉

Before shutting down I made sure the Tak 102 profile for the dome was correct for the 120 Esprit as it is fixed on the same side of the 12″. I then slewed and it centred fine and the dome was in the right position. I tweaked the focus too in preparation for the Messier Marathon with Basingstoke Astronomical Society this Friday.

Viewing Report 20th March 2022

20:00 – 21:52

After a glorious sunny Sunday where I spent a lot of the day building the new decking for the Summerhouse for my wife, I planned on a short, sharp observing session until 9:30pm (turned out to be nearer to 10pm) starting after dinner at 7pm. In preparation for this I setup the 22″ dob on the observing patio and opened the dome for the 12″ to start to cool down whilst I ate with the family. My daughter and her boyfriend’s famous pasta dish was for dinner tonight which was great 😋

22″ Obsession ready to go

With dinner finished I headed to the sofa and set the 12″ running on NGC 1999 in Luminance so I could then go out and observer with the 22″. It took a while to do the usual, sync and centre on a star, get the focus right and set the imaging run up. But after 45 mins I was then able to wrap up warm and head outside.

I have read an article in Sky and Telescope of visually observing NGC 1999. With that information I opened my newly acquired Interstellarum Deep Sky Atlas to look at the area where NGC 1999 resided. I then referred to the Deep Sky Guide to look at the photos and drawings of the objects in the vicinity including this reflection nebula.

Deep Sky Atlas

There was a potential to observe two Herbig-Haro (HH) objects but I was afraid the sky would be a little to bright given the Moon rising at 9pm, in order for me to see them. I placed the 31mm Nagler in the 22″ and then dialled in NGC 1999 into the handset and pushed the scope until the numbers for Azimuth and Altitude were as close to zero as possible and looked through the lens. Surprisingly I could see a small fuzzy object with a star embedded to one side just off centre in the eyepiece. I reviewed the star chart on my phone using Sky Safari 4 and confirmed this indeed was NGC 1999. I guess it was just off centre due to my alignment and next time out I will use the 13mm Nagler to centre and align the scope during the initial setup.

31mm Nagler

On looking at the reflection nebula I could tell there was something to one side of it due to the offset nature of the nebula to the star. I changed to the 13mm Nagler and set the Paracorr accordingly back to H from the A setting for the 31mm Nagler. I then recentred using the Argo Navis computer and too a peak through the eyepiece. I could now see a distinct whole in the nebula, but clearly not as good as the Hubble image I had looked at, however it was there. I ten went looking for the two HH objects but I must say I could not confirm them at all. There were two star like objects near by but again looking at my Sky Safari star chart I was not convinced. I would leave this for another night when the Moon was goner and I had cooled the mirror with the fan, that is still not connected to power yet.

For a laugh I then pushed to the Horse Head but could not see it, not surprised given you need a Ha filter. I will buy one.

I then had a quick look at M42 again which is a wonderful sight in the scope. Orion was now getting low with NGC 1999 and M42 at +18 degrees at about 9pm. I then started to pack away the scope to head in doors and look at the 12″ and see how it was doing.

NGC 1999

I reviewed the set of images for NGC 1999 on the 12″ and now due to the altitude I switched and set the scope running on M61 for Luminance and RGB as it is on my list for my Messier wall chart.

At 10pm I left the dome capturing M61 LRGB frames and hit the sack.

M61

Viewing Report 19th March 2022

20:00 – 23:00

So after a successful day at The Practical Astronomy show it was time to head on out to play with my new toys. In this case a new 31mm Nagler Type 5 2″ eyepiece that I purchased from Owen Brazzel.

31mm Nagler

The other aim tonight was to setup the Argo Navis computer to allow me to find objects in the night sky on the 22″. Owen had said to me at the show that this is a must else I would find it difficult to star hop from one object to another without it, which was proved out the last time Bob and I ventured out.

Argo Navis

So I went to the Workshop where the Dob is stored and took it out onto the observing space and started to put it together. It only takes about 15-20 mins and then I set about collimating quickly with the laser collimator.

Laser colimator

Once done I fitted the Paracorr optical corrector followed by the 2″ eyepiece. I then set about putting the cables in for the Argo Navis computer and clipping it onto the side of the 22″.

It took me over an hour to finally setup the computer, in the main I did not RTFM and finally succumbed like all good blokes to reading the manual. It was them extremely straight forward. I had a good look through the menus to familiarise myself, but really the only thing to do was to identify 2 stars. The process for this must be followed for it to work correctly.

So I selected Mode Align Star option form the menu clicking the Enter key, then selected a star using the Dial, in this case BETELGEUSE came up. Now DO NOT PRESS ENTER!! Put the star in the centre of the eyepiece and then press the Enter button. Then WARP= +0.00 (1) should appear briefly meaning 1 star is aligned. Now using the Dial select another star (in this case SIRIUS) and repeat. Once you select ENTER then you see WARP= +0.00 (A) which means aligned. Now you are good to go and use the catalogue to tour the night sky.

I then selected some objects including M42, which looked lovely but low, the double double which really allowed the eyepiece quality to show off its abilities. I went round few other objects and Luke came out too, In fact to be fair he helped me get the computer working and then we shared the views in the scope. It was a good evening. We then between us took the scope apart and put it back in it’s home.

Obsession in use with the light on for the photo of course 🙂

Exoplanet HAT-P-66b (25/2/2022) – IMT3 + Visual with 22″ Dob!

21:24 – 03:00

So for the first time in a while the observatory is back up and running and it’s Friday and it’s clear! So Bob came over and we set about imaging an exoplanet and getting the 22″ Obsession dob out of the warmroom where it had been sitting since November, and moving it down to the observing plateau by the IMT3b observatory.

A few nights back, during the week I had managed to get the 12″ in the dome balanced with my original 2004, 5″ Skywatcher 130 reflector as a guide scope and the Skywatcher Esprit 120 ED on the other side. The 5″ now guides, however the QHY camera on the 120 is not currently connecting so I need to investigate. Also the MEII hand controller is sending erroneous signals to the mount so I needed to disconnect as it kept saying ‘joysticking’ and moving the mount. Unfortunately the 5″ was pointing at the dowel so I used off-axis guiding instead.

Blue 5″ scope pointing at the dome

I selected an exoplanet on the Exoclock website, and set about imaging. Unfortunately due to my profile on exoclock being set to UTC +1 and me not noticing I started 45 mins into the transit! I will now use this as a test run and process the data and submit.

Exoplanet schedule

So first up was moving the 22″ down to the plateau so it could cool down. I had planned on taking it back up to the Warmroom, however it is so heavy I have decided to keep it in the workshop. I will make sure I connect the electric up for the workshop this weekend thus allowing for the new dehumidifier to be switched on and operational.

22″ UC Obsession telescope

Whilst the 22″ cooled down, we went back indoors and setup the 12″ in the observatory to image the transit. The focus point was 25,000 and I made sure I selected the Red filter. I slewed and aligned on the exoplanets host star and then settled on a 5min exposure due to the low brightness. I started imaging at 9:24pm, the transit start was 8:45pm 🙁

HAT-P-66 b

Whilst imaging continued we went outside and set about setting up the 22″. Things I learnt from this first public outing were;

  1. Remember to put the shroud on
  2. Remember to pub the secondary shield on
  3. Remember to collimate the secondary first
  4. Remember to fit the Telrad dew heater
  5. Remember to find a 40mm eyepiece
  6. Remember to download and print the Parracor chart
  7. Remember to set the Parracor to the correct setting
  8. Remember to buy a 12v battery powered hair dryer
  9. Remember to buy a secondary dew heater

We collimated the scope and then went on to look at M42 which with the 24mm Panoptic looked great. We then slewed to M45 the Pleiades but of course the open cluster is too wide for the for I was using. Then we tried find M81 and M82 but I found this difficult due to the lack of bright stars in the area. So instead we went to the Trio in Leo.

M65 and M66 looked fairly bright and you could make out the dark dust lane in M65. NGC 3628 was visible with direct vision, however with adverted vision it was much easier. It appeared very long and thin.

We had to use s hairdryer on the secondary a lot. The Telrad also dewed up as did the eyepiece. The primary mirror was fine, although with direct headtorch light looked like it was dewed, but upon inspection this was not the case.

At midnight it was time to pack up before I had to do a meridian flip. Bob helped my dismantle the 22″ dob and then place inside the workshop. I could not move it into the room without dismantling which was a shame. It is far too tall.

Once done, Bob headed off home for a well deserved rest and to warm up due to it being 0℃. I headed indoors and performed the meridian flip at 0:23am.

I then left the observatory running until 2:30am whilst I wrote this blog, had a glass of wine and ate an easter egg 🙂 I also went back out to the workshop to tidy up the telescope eyepieces etc that I had not put away.

At 3pm I went on to take luminance for M85 that I needed to complement the RGB data I had processed. I let this running and went to bed.

M85 Lum 300s

I took flats and dark flats in the morning, although I had to increase the exposure to get the required level for the flats. This is strange as they should be the same. I am now concerned the camera my be playing up and causing an issue. I will investigate.