Attilla Danko's
Boring Home Page

Just the Links

Off Site:
BigDob
Ottawa Astronomy Friends

Clear Dark Sky / Clear Sky Charts Update 2026-01-20

Thank you VERY much to all the people who sent concrete suggestions in to the problems below. Although I couldn't quickly figure out how to integrate Cloudflare into the current system which uses EasyDNS to loadshare already, I did use the Cloudflare idea of "Prove you are human", to poison f.php which was the target of the hackers. It had the same name as an insecure wordpress extension, so hackers thought they were close to hitting the gold mine. Except they couldn't get it to do what they wanted, but they kept trying. f.php, which was used to dynamically generate the crosshairs on the map for people looking at detailed forecasts, now gives a static page asking people to prove they are human by editting the url to get to the newly renamed real code. And does a sleep to annoy hacker-bots. Sorry, I know this will annoy real people too. but I decided it was either inconvenience people for a little bit, or shut the site down. Not only was one hoster asking me to upgrade because the site was consuming too many CPU cycles, the other hoster kept falling over because just the access logs were filling up the disk space so fast I had to clear them out daily, or the updates would fail from lack of space. And EasyDNS was charging extra money because of all the excess name lookups that were happening. With the poison in the .php, all this has dropped off to the much more normal traffic levels. Whew!

This whole experience has accelerated my intention to hand the site off. Fortunately, some kind and generous people stepped forward to say they were willing (and showed they were able) to take over the site. Several more people have offered general help. Anyone who has offered/will offer help, will be passed along to the people who will take the site over. I just hope this happens before the next thing goes wrong.

Clear Dark Sky / Clear Sky Charts Update 2025-12-30

For those who still don't know Attilla Danko died 2024-11-28.
If you have any stories/anecdotes/memories of Attilla, please email them to danko@pobox.com .

An ashes scattering ceremony was held 2025-09-28 on what would have been his 70th birthday, at the North Frontenac Astronomy Park. Ironically, just the weekend before, they held their first overnight star party. He had been really looking forward to them holding such star parties so he could bring a big scope of his, and justify the several hours of setup and several more of teardown.

Chart forecast support update 2025-12-30.

I had thought that supporting the charts would be a matter of reading the code, understanding it, and fixing any bugs that came along, as well as updating it for new charts.

I was wrong.

By far the majority of the time has been spent with server issues.
I am not a webmaster expert, nor do I wish to beome one. I wanted to keep this site going as a tribute to Attilla, but I also overestimated what I could do.
Grieving sucks a huge amount of my energy: I can only do about 1/3 to 1/2 of what I used to be able to accomplish in any given time. And my workload has trippled: Not only do I have my own chores, but also Attilla's and OMG all the work trying to settle the estate, which might take another year. Working on the site retriggers my grief big-time. To relieve my pain, I've droppped down to monitoring email just once every week or two, instead of the several times a day that Attilla did. I know the site gave him a lot of pleasure and a sense of purpose, but it isn't doing this for me.

I need to get help with this site, or I will have to take it down.

As of 2025-12-30, the important technical problems I need to get solved are:

• How to get a more sustainable SSL certificate strategy. I have finally figure out how to get Hostgator certificates to work, but it requires time spread over several days every few months. I need strategy that does not require so much manual effort on my part. Because of the load - and because hosts stop working every so often, Attilla had two or three hosting companies, and used EasyDNS to load-share between the hosters, and to quickly disable a site that wasn't working until it could be fixed. Let's Encrypt requires that all the DNS records point only at the hoster that's trying to get such a certificate or renew. If it tries to renew while the load is being shared - or everything is pointing to the other hoster because of an issue, the certificate won't renew. And wildcards aren't allowed because the DNS is being done through a service other than on the hoster. If the pointing and repointing isn't done at the right time, the certificate breaks, and many people complain. The pointing has to be done to fix it, resulting in even more people complaining. I did ask one of the hosters about a commercial SSL certificate. These only last for 1 year, and at the year end, the same ringamarole has to happen. But with more human intervention and costing money for the priviledge.
• Here's the biggie As of late 2025, at least one of the servers is being attacked by hackers. The normally 85K/mo unique IP address accesses through the public website, is suddenly 10 times higher. And the private URL used for testing and updating, that should only get at most 1K/day, is at 1,400,000. The access logs alone, are filling up the server's disk space so quickly, that it runs out of space causing updates to fail until I manually download the logs and clear out some space.
  
  How do I know they're hackers? Because most of the queries are coming from Brazil, Russia, India and other non-North American sites, and they aren't accessing charts, but are looking for Wordpress software that... the site doesn't use and doesn't have. My hosters/ISPs are sending me automated messages that I'm exceed the resources for the plan I'm on, and should upgrade. I'd upgrade for legitimate increases in use, but not to feed hackers. And they're sophisticated enough to be querying with unique IPs, so I can't just block an IP or two.
  
  I don't know how to solve what is turning into a distributed denial of service attack. And without help, I will resort to Attilla's original wishes, and let the site die.

Longer term:I need to find a team of people with time, ability (and permission from their boss) to help with the site.

ClearDarkSky is living on borrowed time and needs a new maintainer (or maintainers). Someone who has time and energy (and doesn't burst into tears whenever they try to read the code). Someone who enjoys developing code and playing with websites. Who knows astronomy and what matters to astronomers in a forecast for the middle of nowhere. Already knowing python would be a bonus, but being willing and able to learn enough about it to read the code is good enough. Ideally, someone with enough space to host a server or two. And who is willing to become the public face for the site. Ideally someone who isn't deaf (like me), who can talk to hoster support lines on the phone. You will need to have high-speed internet. And if you work for another employer, you will need permission to retain ownership of this intellectual property.

This would be ideal for someone recently retired who wants to contribute to the community.

How much time will it take? Time to check emails ideally daily and code up new chart requests, which currently happen about once/week on average. Plan on 20-30 min per request. And time to deal with issues that arise. Plan on such an issue every couple of months, taking 4-8 hours each (assuming the hacking issue gets solved). Plus however much time it will take to change the code when underlying data sources change. Plus however much time you want to spend to convert/upgrade the site to suit you. If you want to accept requests/contracts for archived data analyses, that too will take time, but you can negotiate the rate with the requestor. The site is currently demonetized, but with the right maintainer(s), sponsorships and advertising could easily be resumed, but tracking them for tax purposes will take time.

If you know anyone who is willing and able to do this, please have them e-mail danko@pobox.com . I - and all the users of cleardarksky.com - thank you.

Below is my original post about what I had intended to do for the site.

Update Late 2024

Also, many of you are wondering what will happen to the Clear Sky Chart Forecasts, so I thought I'd share what I know.

Attilla wrote the code to do forecasts for himself, but then decided to share similar forecasts with anyone who wanted them. He made a hobby out of looking for what cloud forecasts he could find and trying to compare their accuracy. He'd find other useful links and add them to forecasts - but sometimes only the sponsored forecasts.

He organized a letter writing campaign that helped Environment Canada decide to make the cloud forecasts a formal product - he would never have been able to do these charts without Allan Rahill's invaluable contribution, which we will forever be grateful for.

He fully expected that within a few years of him starting his forecasts, someone else would do a better job and take over forecasting.

And yet... people kept wanting his forecasts.

But because he always expected someone else to put him out of business, he never productized his code.

When he first became ill, I asked him about the succession plan for cleardarksky and he told me it should die with him. I didn't agree. I asked him to consider letting someone else take over, but he wasn't really trying hard to make that happen.

It was only while he was in hospital early October 2024, that he finally agreed to show me how to do the most basic things, like add a sponsorship or a new chart. Since then, I've been doing that maintenance. (I'm the "colleagues" he refers to below - we never did get anyone else on board). We did agree that to reduce the workload, the site should be demonetized, which happened late November.

However, he was too ill to give me a full guided tour of the code. And looking at the folders, there are still personal financial files and emails intertwingled, so I can't just zip it and offer the code to other people. Because of this, please don't ask me how you can help: I haven't been able to think of anything appropriate that other people can do at this time, and it sucks to have to tell people this one by one.

I can - and will - continue to create new charts (but maybe not as fast as Attilla did). The generation code is automated, so as long as nothing goes wrong (or changes), forecasts will continue to be created. But I have a complex estate to settle now, and it will be quite some time before I will have time (and energy) to try to sort through 20+ years/100kloc of unproductized code development (including stale files and other surprises) and learn enough python to figure out how to package the code for handoff. Hopefully I'll be able to get to this before something breaks, but no guarantees.

Clear Dark Sky forecast design summary

Several people have come forward to express interest in helping take over Clear Sky Chart code long term. Some think that it's a simple matter of tapping into Enviroment Canada's data, so I thought I'd try to capture some of the design considerations in the current implementation of cleardarksky forecasts.
When Attilla started the forecasts, only images were available, so that's what his code uses. Since then, Environment Canada has implemented a datamart. Despite now being retired, Allan Rahil has kindly supplied the following information:

All images are available on this Environment Canada website  https://weather.gc.ca/astro/index_e.html

For the sustainability of astronomical weather forecasts, before [Allan's] retirement, 
all datasets were converted to grib data and are available in the datamart. 
Everyone is therefore free to produce images provided that the source (ECCC) is mentioned. 
Using GRIB data has advantages over images because they are available sooner after the model runs 
(4 times per day) and there are also online interpolation tools available. 
Here are the websites to extract the data;

https://dd.alpha.meteo.gc.ca/model_gem_regional/astronomy/grib2/
https://dd.weather.gc.ca/model_gem_regional/10km/grib2/

TCDC= Total cloud cover
TMP = temperature (2 meter)
SEEI = seeing
TRSP = transparency
WSPD = total wind speed (10 meter)
WDIR = Wind direction
RH = Relative Humidity (2 meter)

You will find information on all variables from these sites;
https://eccc-msc.github.io/open-data/msc-data/nwp_rdps/readme_astro-rdps-datamart-alpha_en/#liste-des-variables
https://eccc-msc.github.io/open-data/msc-data/nwp_rdps/readme_rdps-datamart_en/

You will find information on gib2 format, decoding, processing, interpolation and visualization on this site:

https://eccc-msc.github.io/open-data/msc-data/readme_grib_en/

It seems very likely that clear sky charts should convert to the datamart, but this wasn't something Attilla had done.

Instead, his code monitored the Environment Canada website, for when the new images would be available.
When they were, he'd download them all (just once per cycle, but hundreds of images), and generate the 6000-7000 forecasts from that data. He didn't want to overload the Environment Canada servers by requesting all the images for every chart, which is one of the reasons he never implemented a dynamic chart location feature. When only an incomplete set of data was available, the generating code would fall back to older forecasts. Some of the forecasts would get released in tranches. There was code to handle that, and regenerate if a subsequent tranch showed up before the next planned generation cycle.

He'd also grab the forecasts from the European forecasts (ECMWF) which he would get from the Norwegian site, met.no . (Fun fact: the Norwegian word for "cloudy" is "skyet". Get Google to pronounce it for you. NSFW). This is done on a one by one basis, so to keep the server load low, it was originally only done for sponsored charts.

He also obtained copies of the latest light pollution surveys.

Anything image based, would of course need a transform to identify which pixel in the map image corresponded to the longitude and latitude for any given chart. Any time the map image format changed, he'd reverse engineer the transform needed. And when maps had political boundaries drawn on them right ontop the best pixel to use (obscuring the forecast data for that site), he'd calculate the second best pixel to use, and use that instead.

His code would generate large format charts, and smaller versions suitable for embedding. With the right incantations to avoid caching, many clubs embed the forecast for their favourite observing location on their website. This is a popular feature that would be good to maintain in any future iteration.

Now it starts to get tricky: MUCH of the code complexity was to create reliability. Any time the forecasts went down (which they did, like clockwork every single time we took a trip until he refused to travel anymore), he's get lots of unhappy-users email.

So he implemented code (version controlled with SVN) to support multiple computers generating the charts in loadsharing and failover mode. And there are three internet service providers with three different last mile techs (fibre, copper phone line, and cable) to two physical locations for redundancy. A UPS in the house doesn't help enough when the power outage takes down your ISP too.

And there are currently two but previously up to three webhosters, with a dns server to do loadsharing or manually allow disabling a broken hoster. The contents of the sites are refreshed every time the forecasts are updated, with complex code to account for the fact the different hosters require different directory structures. And code to tar/extract files, because hosters have limits on how many files you can upload. And code - lots of it - to support sponsorships.

For the sponsored charts, he'd archive forecasts to allow a history.

He also calculated chart usage stats, which, along with sponsorship information would determine the order in which he generated the charts. And... to delete charts that weren't used enough.

He also linked Near-Realtime Satellite Imagery, Sun & Moon Data, a Road Map, a Topo Map, Civil Weather, Satellite prediction, and a light pollution map. Each of those would change every few years, which meant that every year, something would need updating to accomodate whatever changed.

To help him assess the quality of forecasts, he also had code (not visible publicly) to grab copies of cloud covers reported in metars (oktas). He'd compare those to cloud model forecasts for the airport locations, to assess whether including the forecast was worth it. He used that to decide to include the European model, but not the NOAA cloud model.

I'm sure I've forgotten a number of the design details, but the above list should give you a good idea of what's involved.

-Ingrid
PS. Yes, I'm still adding new charts as requests come in. PPS. Yes, I know the smoke maps are broken.

PPPS. And yes, the site is still having technical issues that jeopardize it continuing, currently around certificates and server load, and the maintainer's inability to read code due to excess eye moisture triggered by grief. Ideally, someone who has both computers and astronomy as hobbies would be willing to take it over.

I've left Attilla's original description about the charts here below.

Colleagues

Me

I never had a use for a personal website until I heard about the computer language Python. I figured any language named after Monty Python's Flying Circus had to be cool. But to learn Python, I needed a problem to write code for. I found it tedious to add 5 in my head to convert UTC to EST in using the astronomy forecast maps at CMC, so I started writing code.

Next thing I know, I'm writing optical character recognition code, reverse-engineering map transforms, writing javascripts and web databases, writing failover and load-sharing code for windows and generating Clear Sky Charts. Because of the very cool numerical model Allan Rahill (of CMC) wrote, the Clear Sky Charts turn out to be just about the most accurate forecasting device for astronomers. Then word got around.

I'm generating clear sky charts for >3000+ observatories and observing sites in North America and having an absolute blast. I wish knew how to turn clear sky charts into a livelihood so I could do it full time.

There are a few other things on this website that largely came about from the charts:

• I wrote the Ottawa Astronomy Weather page to explain to my fellow observers in Ottawa why the local forecasts were so bad and where to get real astronomer's forecasts.

• I wrote the Seeing Observations Database so Allan Rahill could get real data on astronomical seeing in order to tune his numerical seeing model.

• I answer a lot of email. In a pitiful attempt to stem the flood, here are some of the questions i've answered:
  • How to buy a telescope.

And then there is just plain sillyness:

• Many fine nights observing with buddies, who were also Monty Python fans, cause me to write (in a moment of weakness), a translation of the classic monty python sketch, The Four Observers, into the language of amateur astronomers.

• Many fine nights observing with buddies who like to express themselves led me to realize that one could rate astronomical views by listening to people swear at the eyepiece. In another moment of weakness (is there a pattern here?), I wrote the Expletive Scale (Not suitable for kids) of astronomical observations.

I've come to appreciate the social aspects of the astronomy hobby. So I deliberately created a few places where astronomers could yak:

• The BigDob (big dobsonian telescopes) mailing list.
• The Astronomy Weather mailing list.
• The Ottawa Astronomy FriendS mailing list for by observing buddies who say Ni!.
• My rabid-opinions page on telescopes and weather on the OAFs wiki.