Post by Keith (Southend)GPost by Auric__Post by Keith (Southend)GNot sure if this is relavant, but the text file I save with this data
is always called "ogimet.txt".
Do you add the new data to the existing text file, or do you replace it?
(FYI, I switched from a batch file to FreeBASIC. I'm a truly terrible
VB.Net programmer, and I hit a brick wall in the batch. Doing it in
FreeBASIC will let me give you a single program to compile, instead of
"do this and then do that and then do this third thing and then..."
Results later today or tomorrow, depending.)
I replace it. This file is just a means to an end, if you know what I mean.
I have something else that sorts it, and then I add the sorted data,
which is broken down into hourly files, into, the hourly file(s) I
already have.
It probably sounds confusing, but basically, the data from this site
contains extra locations that I can not get from anywhere else, which I
add to the main files. It just takes me 10 minutes each day to put this
'ogimet.txt' file together and would like to be able to download the
multiple hits straight into one file. Actually, the name of the file
initially is irrelavent.
As I said, this is FreeBASIC, not VB.net. Any attempt at using this in any
version of VB (.Net or otherwise) will give you all sorts of lovely errors.
FreeBASIC is available here:
http://www.freebasic.net/get
(The installer is convenient, but I just use the .7z archive.)
The program is at the end of this post. To compile it, type this at the
command prompt:
fbc ogimet.bas
To function properly, the program needs 2 text files:
- countries.txt
This is a list of the countries you want, 1 country per line, like so:
Afg
Como
Keny
You can use either the short code (from the ogimet.com page's source) or
the country's full name, or any part thereof that makes it unique (i.e.
if you give it "trinidad", it returns Trinidad and Tobago, but if you
give it "united" you'll get UK, US, and UAE).
- dates.txt
This is the start and end date/times for your reports; first line is the
start date and the second is the end. The format looks like this:
2012 01 01 01
2012 01 02 00
Somewhat obviously, it's year-month-day-hour. It doesn't matter what
character is between the fields, as long as they're not all jammed
together. (I used spaces; you may want to use dashes or slashes.)
The program outputs everything into ogimet.txt. Fyi, it can take a while;
your ten minutes might only get cut by a minute or two, depending on what
countries you download. (I told it to grab the above 24 hours for *all*
countries; it took about 14 minutes, and resulted in a 10mb file.)
(In case you're wondering why I switched from batch to FB, it turns out
that ogimet.com sends the text file wrapped in HMTL, with Unix line
endings. I already had a program written to chop text out of <pre> blocks;
it was pretty easy to add downloading and Unix-to-Windows to it.)
If you need any help with this, give a shout. (Stay in this thread so it
catches my attention.)
'-----begin ogimet.bas-----
#lang "fblite"
CONST CR$ = CHR$(13)
CONST LF$ = CHR$(10)
CONST CRLF$ = CR$ & LF$
CONST xFF$ = CHR$(&hFF)
CONST url1$ = "http://www.ogimet.com/display_synopsc.php?lang=en&estado="
SUB REPLACE (baseStr AS STRING, replaceThis AS STRING, withThis AS STRING)
DIM tmpStr1 AS STRING, tmpStr2 AS STRING, tmpLng1 AS LONG
DIM lenIn AS LONG
lenIn = LEN(replaceThis)
IF lenIn < 1 THEN EXIT SUB
tmpStr1 = baseStr
looper1:
tmpLng1 = INSTR(tmpStr1, replaceThis)
IF tmpLng1 THEN
tmpStr2 = tmpStr2 & LEFT$(tmpStr1, tmpLng1 - 1) & withThis
tmpStr1 = MID$(tmpStr1, tmpLng1 + lenIn)
GOTO looper1
ELSE
baseStr = tmpStr2 & tmpStr1
END IF
END SUB
DIM cmd AS STRING, inpStr AS STRING, tmpStr AS STRING, tmpLng AS LONG
DIM URLDownloadToFile AS FUNCTION (BYVAL pCaller AS LONG, _
BYVAL szURL AS zString PTR, _
BYVAL szFileName AS zString PTR, _
BYVAL dwReserved AS LONG, _
BYVAL lpfnCB AS LONG) AS LONG
DIM AS ANY PTR library = DyLibLoad("urlmon.dll")
URLDownloadToFile = DyLibSymbol(library, "URLDownloadToFileA")
OPEN "countries.txt" FOR INPUT AS 1
OPEN "ogimet.txt" FOR OUTPUT AS 2
OPEN "dates.txt" FOR INPUT AS 3
LINE INPUT #3, tmpstr
tmpstr = TRIM$(tmpstr)
yrst$ = LEFT$(tmpstr, 4)
most$ = MID$(tmpstr, 6, 2)
dyst$ = MID$(tmpstr, 9, 2)
hrst$ = RIGHT$(tmpstr, 2)
LINE INPUT #3, tmpstr
tmpstr = TRIM$(tmpstr)
yren$ = LEFT$(tmpstr, 4)
moen$ = MID$(tmpstr, 6, 2)
dyen$ = MID$(tmpstr, 9, 2)
hren$ = RIGHT$(tmpstr, 2)
CLOSE 3
WHILE NOT EOF(1)
LINE INPUT #1, what$
what$ = TRIM$(what$)
IF LEN(what$) THEN
tmpLng = URLDownloadToFile(0, url1 & TRIM$(what$) & _
"&tipo=ALL&ord=REV&nil=SI&fmt=TXT&ano=" & yrst$ & _
"&mes=" & most$ & "&day=" & dyst$ & "&hora=" & hrst$ & _
"&anof=" & yren$ & "&mesf=" & moen$ & "&dayf=" & dyen$ & _
"&horaf=" & hren$, CURDIR$ & "\" & TRIM$(what$), 0, 0)
IF 0 = tmpLng THEN
OPEN what$ FOR BINARY AS 4
inpStr = SPACE$(LOF(4))
GET #4, 1, inpStr
CLOSE 4
REPLACE inpStr, CRLF$, xFF$
REPLACE inpStr, CR$, xFF$
REPLACE inpStr, LF$, xFF$
REPLACE inpStr, xFF$, CRLF$
tmpLng = INSTR(LCASE$(inpStr), "<pre>")
IF tmpLng THEN
inpStr = MID$(inpStr, tmpLng + 5)
ELSE
PRINT #2, inpStr;
GOTO noPre
END IF
tmpLng = INSTR(LCASE$(inpStr), "</pre>")
IF tmpLng THEN inpStr = LEFT$(inpStr, tmpLng - 1)
noPre:
PRINT #2, inpStr;
KILL what$
END IF
END IF
WEND
CLOSE
DyLibFree library
'-----end ogimet.bas-----
--
There's not enough liquor in the world to make me forget this nightmare.