.

.

All techie stuff here
#314511
Regional Atmospheric Soaring Predictions (RASP)

This thread is about sparking interest in creating Blimpmaps, RASP, WingGrams and the like in regions like yours.
- Nearly mandatory for regions that become well known for hosting top level XC Events up to and including FAI Category 1 World Championships;
- And our FAI Olympics: the World Air Games.

See:
RASP BlipMaps for futher Information;
RASP Creation - information for those interested in running RASP for their own region;
RASP Forum - comments and discussion by RASP user/operators.

As you can see, most of the major competition regions in North and South America do not have RASP Services provided for them yet.

Image

]

[youtube]
[/youtube]
Last edited by Fred Wilson on Tue Mar 05, 2013 11:40 pm, edited 3 times in total.
User avatar
By Fred Wilson
#314513
Two of the very interesting latest threads focus on modern technology, computers, Virtual Machines and Cooperative ventures with Atmospheric Faculties at Universities, sharing workload and expenses with our National and State Associations, Local HG/PG Clubs, SAC Sailplane Associations and the MAAC RC Glider's world.

See in particular the two + threads on:
1. The OpenMeteoData project;
2. Installation on 16 physical core/60GB EC2 inst; and
3. Cooperative ventures to Upgrade Dr. Jack's 32 bit architecture to the new 64 bit world
at: http://www.drjack.info/cgi-bin/rasp-forum.cgi
TJ Olney wrote:You need to decide what you want. You need to find informative links as to what you want to do with data from them.

Is a RASP output what you want? You need access to initialization data and someone to run the model and organize its output.

Is it live observations that you want? All are available in many formats. See the list of all weather station's feeds in BC.

Do you have fixed ideas about how the data should be presented? Are the hard to read and obsolete soundings what you want?

http://www.twisterdata.com/index.php may have everything you want.
Last edited by Fred Wilson on Wed Mar 06, 2013 12:48 am, edited 3 times in total.
User avatar
By Fred Wilson
#314514
Andrew wrote:RE: Cooperative ventures to Upgrade Dr. Jack's 32 bit architecture to the new 64 bit world:
"The 4 year old relatively slow quad-core machine that I got really cheap is only capable of running 1 site with what I know so far about gridding and reliability.
It may be that I will learn more in the next few months about how to more efficiently grid and choose boundary conditions,
but so far, not being a meteorologist and not having infinite time my guess is that to cover the interior I'd need a better computer.

I think what would work is a top of the line quad-core Intel i7 with 4-8G of quad-channel very fast memory, as the run speed is mainly memory access dominated.
That together with this beater machine should be able to cover three local regions / valleys. Not sure if I could get another in there.
It would cost < $1500 in my rough estimate (I just need the headless box, I have a monitor, etc here).
I would want to use the your or our club's website for publishing the results if I can arrange it with the guys.
I don't mind hosting the computation server at my house and paying for the internet (about 100 gigabytes a month) and electricity costs out of my pocket.

The other option is for me to just write some scripts to generate output from the 2.5km grid Canadian HRDPS data
(I'm running the RASP at a 0.83 km grid) which covers the west coast (something better than spotwx.com does, for example)
which would take no processing power, but might get us some better wind information for the interior valleys.

The main output from the RASP that is better than the general and aviation weather services is
wind predictions in valleys and across mountains and as well predicted local sounding type information.
There is local CAPE and storm information but without some significant investment in time validating the model against
reality we won't do better than general and aviation forecasts _even_ if they are for areas pretty far from where we are flying.
And, it's only as good in general as the parent HRDPS model, which appears at 24-36 hours out to be about as good as the GFS or the NAM.

Still slowly working on seeing how good the HRDPS data is alone...
- for small valley wind predictions it won't be useful because it barely resolves the valleys that we fly in.
For general XC conditions it's probably fine if we can generate a mapping of the variables to something useful for the visualization scripts.

If I had immediate questions I would post it to the RASP forum as opposed to somewhere else.
Bye! Andrew"
Last edited by Fred Wilson on Thu Mar 07, 2013 6:40 pm, edited 7 times in total.
User avatar
By Fred Wilson
#314515
Andrew wrote:Re: setting up a local Club RASP Page
FYI: Re the comment below, See: TJ Olney's
a) Pacific Northwest RASP BlipMap Service (example) at: http://home.comcast.net/~wxtofly/blipspotWoodside.html
b) Pacific Northwest RASP Service (example) at: http://home.comcast.net/~wxtofly/
_________________

Quoting Andrew:
"I've been recently working on setting up a RASP for Pemberton.
There is no need to add observational feeds to our 24/36 hour runs as all of it has been assimilated into the data which I am pulling
... and by meteorologists not hacks like me.

So far I've been using the (freely available) 2.5km grid data from the Canadian high resolution local area model
(HRDPS: http://www.weatheroffice.gc.ca/grib/gri ... _HR_e.html ) which runs on the west coast
with some extra data from NCEP Model Analysis / NOAA for some of the less relevant parameters that the WRF model needs.

Peter managed to find us a cheap machine to do the runs on, a quad-core last generation processor, and I am providing internet, electricity, and hours.
It is about 4x slower than a top of the line server with quad-channel memory (approx cost $2000).
So far a 100x100km at 1km grid Pemberton model takes 5 hours to do a 24 hour forecast
(30 minutes of 1GB of data retrieval and assimilation and 4:30 of run time).
If I start getting interesting results I will move it over to my personal gaming machine
(which gets little use during the flying season) which will be about 2-3x faster (which means we could do coverage of the interior as well).
TJ Olney manages to run a 4km outer domain over the whole pacific north west and then a handful of
small 1km type grids local around interesting regions on a single machine with reasonable run times.
Maybe I can shrink the pemberton domain, but my gut says I need to expand it so the boundaries are
outside the coast mountains to remove the boundary "noise" due to topography mismatches between
models which will require more processing power... in the end my hopes for accuracy may be overly
optimistic anyway, so I wouldn't throw money at it yet. Later maybe we can offset my electricity / internet bills :)

The reason I'm not done yet (it's been a whole two weeks!) is twofold, first I am using a modern version of the weather models
(WRFV3.4) and second I am trying to use the HRDPS input data, both instead of the out of the box GFS/NAM solution by
Dr. Jack Glendening (though I would say a fair assessment of what comes out of the RASP box would be, appropriately,
a Jack in the Box). Oh, and third, I only have evenings to work on this :)
Another not out of the box solution is the google-maps based RASP visualization I'd like to get up and running
(see http://rasp.inn.leedsmet.ac.uk/RASPtabl ... bleGM.html). I won't forget TJ's windgrams either,
which is the easiest way to get an "is it soarable" forecast.

The reason I am fiddling with data input and fine gridding, though, is that I am interested in capturing some of the local valley wind behavior
(for example, the feared Whistler Express) which requires some of the more interesting RASP outputs and likely careful modeling.
Of course, it may be that it is not predictable easily, but local lore has it that it correlates well with Howe Sound wind advisories,
so all I need to capture is whether that wind can move up the Whistler valley into the Pemberton valley.
(Just in case I fooled you with confidence, here's another reminder that I am not a meteorologist).
Generally, it's in support of some of my XC flying goals :)

TJ -- thanks for your input... I'll be back with real questions once I believe my model is giving reasonable results
(which will probably be on Tuesday or so since so far all it has been doing is raining and snowing up there which isn't particularly interesting). Bye! Andrew"
Last edited by Fred Wilson on Wed Mar 06, 2013 12:22 am, edited 1 time in total.
User avatar
By Fred Wilson
#314516
We are considering building a dedicated PC / Server to host RASPs for a considerable number of exceptionally good XC Regions around here.
All have repeatedly held many, many top level XC Events, including many Canadian National Championships

1. TJ Olney: What would you recommend in the way of a home built Dedicated RASP Server?

2. How fast an internet would you recommend?
- I am about to switch to a new provider. Many options, the faster the *pricier*.

3. You mentioned you might develop a quick install CD at
http://www.westcoastsoaringclub.com/for ... php?t=5740
and / or at: http://www.westcoastsoaringclub.com/for ... php?t=5087

Are we in luck? Tnx
TJ Olney wrote:If you are into building a computer as a dedicated RASP Server, you should talk to Peter and Andrew who are building a new box at the new improved 64 Bit level of RASP for pemberton.
"Their choice to go new and improved means that there are many bugs to work out, but in the process, they learned that
there is 2.5km canadian data available. It is good enough to not even have to process. I got good results plotting from it directly.
There are many things to work out there too, but immediate decisions for anything other than the fastest and the best should probably wait.

As to connection, the system needs to download about 100mb of data for each run. If it is also the server,
then it only uploads what users ask for, which is fairly modest. I think any DSL or better link could handle the traffic.
My plots are not hosted on the machine that runs it so Ihave to upload everything.
I upload 65MB of plots for each region that I cover, three times a day. That at times nearly chokes my internet connection.

If you are going to run the models, you want as fast a machine as you can afford with as much RAM as you can afford -- 4GB RAM minimum.
Just plotting takes significant CPU time, but if you are only plotting, the machine doesn't need to be so buff.

The server doesn't need much power at all. It could run off any old discarded post 2002 machine.
The machine that I use as my scraper machine is a 386 and it is never overworked.

If you have a nix guru, it will be easy. If you have only stumblers like me, they will face lots of learning.
The ideal case is a nix guru who is also a meteorology guru.
Whoever is going to start should start reading at http://www.drjack.info/cgi-bin/WEBBBS/r ... /noframes/
and http://www.drjack.info/twiki/bin/view/RASPop/WebHome

Any scripts that I have made are available at http://home.comcast.net/~wxtofly/rasp_scripts/
All are written for the older version of the underlying WRF software. The windgrams require ncl5.1 or higher.

Good luck! TJ Olney"
Last edited by Fred Wilson on Tue Mar 05, 2013 4:12 am, edited 3 times in total.
User avatar
By Fred Wilson
#314517
Don Herres - the new FlyOK.ca Webmaster wrote:Re: Cooperative projects with HG & PG clubs, Sail Plane clubs, plus partnering with some University Weather Faculties
T.J Olney mentioned that "Peter and Andrew" may be working on building a server / virtual machine to host a BC RASP service for the Pemby region. If so, could we talk? Maybe slide a little money your way?

We are in discussions about doing the same: Building a server or virtual machine to host RASP for:
. .a) Pemberton,
. .b) Kamloops Region,
. .c) North Okanagan,
. .d) South Okanagan - Chelan,
. .e) The Kootenays (Nelson - Spokane) coverage areas.
. ..f) Golden to Wasa (2013 Cdn Hang Gliding Nats)
. . .(and finally maybe tossing in a one month special for Yamaska PQ - for the PG Nats.)

One option is to do it as a cooperative project BC and NW Washington HG & PG clubs, Sail Plane clubs,
using the Spokane data feed http://www.wrh.noaa.gov/otx/avn.php and https://www.facebook.com/US.NationalWea ... pokane.gov
See: http://www.sac.ca/index.php?option=com_ ... acificzone

The Spokane Sailplane Clubs have services such as Soaring Forecast http://www.soarforecast.com/ in place already
- which I believe are pumped out by http://www.atmos.washington.edu/
http://www.spokanesoaring.org/ + http://www.evergreensoaring.info/home + http://www.ieqf.org/

There may be other feeds beyond soundings for Prince George , Kelowna , Port Hardy , Spokane , Quillayute and Edmonton .
http://weather.uwyo.edu/upperair/sounding.html
So Environment Canada could also be approached ( fee based: even NAV Canada has to pay !)
http://www.ec.gc.ca/default.asp?lang=En ... 4545-1#p-y

Plus there may be opportunities to avoid fee based weather information by partnering
with some University Weather Faculties who may consider us for a joint venture.
Examples: Spokane Uni http://weather.wsu.edu/awn.php + UBC: http://www.eos.ubc.ca/
+ UAB: http://easweb.eas.ualberta.ca/page/Atmo ... _Sciences/

Finally:
One item we are keeping an eye cocked out for is the BC Wildfire Service Weather feed at: http://bcwildfire.ca/weather/stations.htm

We discussed taking this as motion to the BCHPA : to obtain this feed. (They might consider chipping in financially.)
- User groups reportedly have to apply for the data feed, which is only pulled during the active flying season.

Their Interactive Map helps you isolate the 260 locations. http://webmaps.gov.bc.ca/imf5/imf.jsp?site=protection

Paul Emmett, MA Fire Weather Forecaster: BC Wildfire Management Branch http://www.bcwildfire.ca
4000 Airport Drive, Kamloops, BC V2B 7X2 - 250-554-5517 (Office) 250-371-7459 (Cell) 250-376-9732 (Fax)

- Washington state has a similar Wildfire Service: http://www.dnr.wa.gov/RecreationEducati ... /Home.aspx

NAV Canada help desk 1-800-876-4693

Meteorological data from several agencies is available here: http://pacificclimate.org/tools-and-data/pcds-portal

Tnx and Yours: Fred Wilson
Last edited by Fred Wilson on Wed Mar 06, 2013 12:23 am, edited 1 time in total.
User avatar
By AlC
#314631
Wow, lots of posts and points to cover.

I've been running the local (Southern California) RASP model for a couple of years http://www.soaringpredictor.info/raspta ... table.html

My "entire region" version covers Mexico to Las Vegas. Just upgraded last fall to an i7 with a CUDA video card. The computer was $450 off Craig's List and the CUDA card was $40. (So you don't need to spend $2 grand...).

If someone has completed a 'brain transplant' to get WRF3.4 running in the RASP framework, I'm sure everyone would love to know. I've seen 3.x compared to 2.x and the results are not markedly better, but the community needs the upgrade path for the long term. (I have run V3.2 as a stand-alone).

If you have "2.5km Canadian data available" then go with it! That's plenty good for XC planning.

I upload my plots to a webhost (GreenGeeks) for $130 for 3 years. Unlimited space and traffic, though they will throttle your I/O if you impact other users.

I'm running the GoogleMaps visualizer and have helped others get it going. Paul Scorer is the originator and responds to posts on the RASP Forum.

Good luck to all and let me know if you need some help.

- Alan
User avatar
By Fred Wilson
#314640
See our new RASP Wiki.
AlC wrote:If you have "2.5km Canadian data available" then go with it! That's plenty good for XC planning. - Alan
Question is: Is that good enoufh for FAI Category 1 and 2 Competitions? That is the real question.

PS
This is ? Andrew's first kick at the can at RASP.
He and Peter are rebuilding Dr Jack's RASP from 32 to 64 bit architeture at the same time.

Would that account for projected cost of at least a couple of thousand dollars for the RASP Server computer?
- Possibly a home built one? (I haven't asked yet.)
Last edited by Fred Wilson on Fri Mar 08, 2013 5:53 am, edited 1 time in total.
User avatar
By AlC
#314651
Fred:

The 64-bit version is pretty common, I'm running that myself. Canadian prices may be a bit higher, but $2,000 would be a pretty big premium. If I bought new it would have been about $1000 and twice as fast (I7 processors have gotten quicker).

The 2.5km grid is fine. We're really using RASP to reduce a 12km grid to 4km or 1.5km. Once you get below a 1km grid there is too much noise to be all that useful. Comps cover a lot of ground and a really tight grid is not needed. TJ has the scripts to go straight to a plot (or re-plot an existing run). If you have 2.5km data available you can save 95% of the processing time and go straight to the plots.

Mark:

No sweat on Ord, and you've got data all the way to the Owens.

- Alan
Last edited by AlC on Wed Mar 06, 2013 12:53 am, edited 1 time in total.
User avatar
By AlC
#314653
Fred:

I should have added that if you only need a model for the comp duration, the EC2 implementation would be a really good option. You could build the RASP model on almost anything (I built my first one on a virtual machine on a dual-core XP machine. The VM was Linux.), then port the finished model to the cloud and throw processors at it for super-fast processing. Some of the Europeans are doing that for their on-going processing as power costs make it a reasonable choice. Pretty inexpensive if only needed for a limited duration.

- Alan
User avatar
By Fred Wilson
#314654
Tnx for the Info! Keep it coming! We are rank novices as will be all who follow us!

See our new RASP Wiki.

But FYI these are all high pilot population areas, which host frequent events
so our plan is to run these mid April to Mid September every year from now on.
- perhaps with a boost for FAI Cat 1 and 2 Events etc. - HG, PG, Sailplane, RC Gliders...
AlC wrote:Fred:
I should have added that if you only need a model for the comp duration, the EC2 implementation would be a really good option. You could build the RASP model on almost anything (I built my first one on a virtual machine on a dual-core XP machine. The VM was Linux.), then port the finished model to the cloud and throw processors at it for super-fast processing. Some of the Europeans are doing that for their on-going processing as power costs make it a reasonable choice. Pretty inexpensive if only needed for a limited duration. - Alan
See Mark Riches' explanation of how he uses the Regional Atmospheric Soaring Prediction (RASP), looking at
the data and parameters he finds useful and showing what this means for paragliders and hang gliders in the UK.
He shows how to interpret a tephigram and how to pick a good flying day.

See the webcast here: http://www.judithmole.net/blog/?page_id=3205
Last edited by Fred Wilson on Fri Mar 08, 2013 5:54 am, edited 3 times in total.
User avatar
By Fred Wilson
#314693
See our new RASP Wiki.
AlC wrote:Wow, lots of posts and points to cover. I've been running the local (Southern California) RASP model for a couple of years http://www.soaringpredictor.info/raspta ... table.html

My "entire region" version covers Mexico to Las Vegas. Good luck to all and let me know if you need some help.
Alan:
Would you have the capacity to add Valle de Bravo Mexico to your coverage area?
It has hosted up to 10 Paragliding FAI Cat 2 and at least one FAI / CIVL Cat 1 Event over the past decade and a half.

The El Penon Pre Pre Worlds is on right now. Their Pre Worlds is next year and they were the ONLY bidder for the Hang Gliding 2015 Worlds.
(The Rigid Wing worlds bid came from France which would be in 2014.)
The Plenary is long over. We should see the results of the voting posted on the FAI Website this week.

Tnx Fred
Last edited by Fred Wilson on Fri Mar 08, 2013 5:54 am, edited 1 time in total.
User avatar
By AlC
#314753
Fred:

Well it's far (far, far) from my current coverage area. It would have to be a separate model.

- Alan
User avatar
By Fred Wilson
#314755
See our new RASP Wiki.

> Fred: it's far (far, far) from my current coverage area. It would have to be a separate model.

Would your server be able to handle the work load?
Would the website you run it on be able to add another coverage area?

Would you be able to run it or major meets only, shutting it down in the interim?
Or could you do it as a one time, one event thing?
(Good PR, especially if you can at least cover all their Cat 2 (and Cat 1) events.

- Or are we living in a pipe dream, hoping to cover that big an area?
Clinton to Golden BC in the North
Princeton to Cranbook along the BC US Border.
Hoping to go further south instead of that... so covering from well south of Chelan to well SE of Spokane.

Plus as a bonus bit of PR Work:
We are hoping to toss in Yamaska Quebec Canada in addition.

Dream on Fred? If so, what would a "Get Real" coverage area comprise of?
This from a total neophyte here.

Tnx Fred.
Last edited by Fred Wilson on Fri Mar 08, 2013 5:55 am, edited 1 time in total.
User avatar
By AlC
#314789
Elmer created a temporary RASP model for this year's pre-world gliding competition in Argentina http://soaringcafe.com/2012/01/pre-worl ... ay-report/ and http://www.drjack.info/cgi-bin/WEBBBS/r ... /read/5519

I probably could not run a full-time Valle de Bravo Mexico model, but I'm sure one of us could put up a temporary comp site. My webhost could easily handle it.

I'm covering more than half of California (163,700 total sq miles) on my older I7. If your guys purchase a newer machine it will be twice as fast so your model certainly sounds feasible.

- Alan
User avatar
By Fred Wilson
#314885
TJ Olney wrote:On RASP Servers.
The new mother boards no longer have integrated video. This is because folks are using the gpu to do a lot of work.

There is now a version of the ncl graphics language that I use for RASP that makes use of the CUDA in graphics cards.
The processing speed is about the same as on the cpu, but it frees the cpu to do the model crunching while it makes pictures.

The bottlenecks for processing are the i/o to ram and size of ram.
Newer sata3 drives and controllers will give a marginal speed boost, but not likely to make a difference that matters.
Discussions about using ssd drives seem to conclude that they don't do much for RASP as it all happens in RAM anyway.

They do speed up the controlling functions, but again, not enough to really matter.

Looking at the current available add-ons that would be needed to make the machine right for the job, the $1500-$2000 number looks about right.

You really want a raid setup, something I want but haven't figured out yet. I back up daily to another disk and then dismount that disk.

The total quantity of data grows rapidly if you keep it around. I trash it every day.
The new doesn't automatically overwrite the old as it is often being used for longitudinal research.

Running the models out more than one day is of course possible and many do it. I don't bother.
There is so much variation run to run that I can get the big picture information directly from the weather services, and only rely on RASP for the next day.
Even there, there are often dramatic changes from the 0Z run to the 12Z run.
_________________
TJ Olney wrote:On Machine Capabilities and Purchase Prices.
As to machine capabilities see:
http://www.portatech.com/products/categ ... ml?id=1239

Look only at intel processors, as they are fully supported by the fast intel compilers that we can
use for free and are often used by people compiling the underlying weather model programs.

I am running on a machine near the bottom, core 2 quad core with a "speed rating" of 2865

If I were buying a new machine, I would buy the third from the top with a "speed rating" of 11980 i7 6 cores at 3.8ghz
If I were using other people's money, I'd buy the one on the top. :)

My 2008 machine has had a hard drive failure, but that is not a mother board problem.
No other problems running RASP, but a few problems getting sound to work reliably, which has not been a priority.

Bye! TJ Olney
User avatar
By AlC
#316998
Current (Mar 2013) pricing on I7-3770 systems with 8GB memory (Dell or HP) is under $900.

- Alan
User avatar
By AndRand
#336915
I'll give it a bump reposting from other thread...

Image

Friend of mine is providing those forecasts. They are detailed RASP BLIPMAPS in GoogleMaps interface. http://fcst24.com/
There are now some high definition (below 5km weather grid) domains made for testing, competition or sponsored. Here is one - made with 500m grid that predicts airflow in mountains (Wind@10m):

Image

The development of the site continues but if the new areas are to be covered with detailed forecasts some funds need to be gathered.
And the campaign on crowdfunding is/will be on: http://www.indiegogo.com/project/preview/a21b2799 :cheer:


Recently I used detailed winds pattern around high mountain massive to find possible places for launch spots.

Image

For the campaign purpose (and sailplane attempt to fly over Everest http://skyfullofheat.com/news/everest-g ... n-project/ ) there is an example with Mount Everest area :P

Image
#337004
Likewise.

Canada RASP - http://www.canadarasp.com/RASPtable.html now includes the southern half of British Columbia
including the vast majority of the flying area used by Vancouver Island's Island Soaring Society.

NOTE:
The southern range extension of this (Southwest BC HRDPS) includes NW Montana,
through and including the Spokane area to Winatchee Washington and on to Bellingham / Seattle.
(Today's drop in seem to indicate Chelan is also included.)

A new, brilliant service thanks to and from Andrew Berkely and Peter Spears of British Columbia's fabulous West Coast Soaring Club.

Very nice! Have you thought about flying seated? I[…]

Not going to play these silly bobK games. People […]

Don't forget the hook-in check especially before c[…]

I guess since were halfway through the comp, if an[…]