[PIC:] Sorting through millions of numbers

[Kresho]

G'day list.

I have a situation as follows:

A machine will have a mag swipe card reader. The cards are encoded with any
number between 0 and 10,000,000 representing an account number. When a card
is swiped i must store the card number, time and date - 9 byte chunks of
data (24bit card number and 6bytes of date/time). When the card is swiped
again i must recall the same information and can then delete it.

Storing heaps of data is no problem - MMC or similar

However, i see an issue in catering for all these cards. Let's say that a
new person swipes his card every 10 seconds. That's 8640 swipes in a day. So
i must store 8640 chunks of data somewhere. The same person may not swipe
his card again for a number of days or weeks. Suddenly, there are thousands
(millions?) of chunks of stored data.

I can see 2 options:

1) Allocate a linearly determined memory address for each particular card.
eg: card #0 data at address h'00', card #1 data at address h'09', card #XYZ
at address 9*XYZ, etc. I would need 10,000,000 x 9 bytes or approx 90mb of
memory. That's do-able with a 128mb MMC, but rather unelegant.

2) I can store the 9byte chunks as i receive them in a free slot in memory.
When the card is swiped again i go through the stored data, from the start
of memory, till i find a match based on the card number, and then i can
access the time/date for that card and delete it leaving a free slot again.
However, if that card is the 1 millionth entry in this memory, it will take
considerable time to find that card number. Reading a million RAM locations
wouldn't be that bad (seconds) but accessing an MMC takes a bit of time and
i can see the time blowing out of all proportions.

Does anyone have any ideas how this could be done in any way other than
option 1 above? My biggest reason for not liking this option is that there
is talk of upscaling the number of possible card numbers to 1x10^12, which
suddenly makes any MMC (or other memory for that matter) look useless.

Rgds,

Kresho Sprem

--
http://www.piclist.com hint: PICList Posts must start with ONE topic:
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

[Ake Hedman]

You can use a B-tree ( http://www.bluerwhite.org/btree/ ) or a double
linked list for your storage.

/Ake

-----Ursprungligt meddelande-----
Från: pic microcontroller discussion list
[mailto:PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org]För Kresho
Skickat: den 21 april 2004 18:14
Till: PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org
Ämne: [PIC:] Sorting through millions of numbers


G'day list.

I have a situation as follows:

A machine will have a mag swipe card reader. The cards are encoded with
any
number between 0 and 10,000,000 representing an account number. When a
card
is swiped i must store the card number, time and date - 9 byte chunks of
data (24bit card number and 6bytes of date/time). When the card is
swiped
again i must recall the same information and can then delete it.

Storing heaps of data is no problem - MMC or similar

However, i see an issue in catering for all these cards. Let's say that
a
new person swipes his card every 10 seconds. That's 8640 swipes in a
day. So
i must store 8640 chunks of data somewhere. The same person may not
swipe
his card again for a number of days or weeks. Suddenly, there are
thousands
(millions?) of chunks of stored data.

I can see 2 options:

1) Allocate a linearly determined memory address for each particular
card.
eg: card #0 data at address h'00', card #1 data at address h'09', card
#XYZ
at address 9*XYZ, etc. I would need 10,000,000 x 9 bytes or approx 90mb
of
memory. That's do-able with a 128mb MMC, but rather unelegant.

2) I can store the 9byte chunks as i receive them in a free slot in
memory.
When the card is swiped again i go through the stored data, from the
start
of memory, till i find a match based on the card number, and then i can
access the time/date for that card and delete it leaving a free slot
again.
However, if that card is the 1 millionth entry in this memory, it will
take
considerable time to find that card number. Reading a million RAM
locations
wouldn't be that bad (seconds) but accessing an MMC takes a bit of time
and
i can see the time blowing out of all proportions.

Does anyone have any ideas how this could be done in any way other than
option 1 above? My biggest reason for not liking this option is that
there
is talk of upscaling the number of possible card numbers to 1x10^12,
which
suddenly makes any MMC (or other memory for that matter) look useless.

Rgds,

Kresho Sprem

--
http://www.piclist.com hint: PICList Posts must start with ONE topic:
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

--
http://www.piclist.com hint: PICList Posts must start with ONE topic:
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

[Kresho]

Thanks for all the replies.

Post by Russell McMahon
I assume this is for something comparable to a car wash system -
you buy a
"ticket" which is recorded in the system and it is removed once utilised.

Good guess but not quite. What it's for is a snack promotion. People will
receive a card encoded with a number. This number is stored with the
time/date as previously specified. Then the card holder swipes their card on
a vending machine to get their free snack. There will obviously be a limited
time for them to redeem said snack, and that's why the date is required.

Post by Russell McMahon
You didn't mentioned "PIC" one single time, but I'd guess that
that's what you'd like to use, right ? :-)

Yes, or something similar. I'm dabbling with some hitachi H8s at the moment
which might be a better fit for this application.

The b-tree and hashing schemes seem like a possibility. However, I think
Adam's response with his practical analysis is probably fitting here. If the
limited time for redemption is kept short enough I can just go through the
data every now and then and delete expired entries and keep the list lean.

If i come up with another solution i'll keep you posted.

Rgds,

Kresho Sprem

--
http://www.piclist.com hint: PICList Posts must start with ONE topic:
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

[M. Adam Davis]

If it's a snack promotion that must be used in a specified amount of
time, I'd go about it a completely different way:

Encode the card number and the date it has to be used by on the card.
You might want to encrypt this information so it can't be easily duplicated.

Reserve one bit of memory for each possible card number on the memory
card. If the card has been used, the bit will be set to one. If the
card is too late (date) then don't accept it.

Here's the trick: You only need a few thousand numbers.

The cards expire on a regular basis.
The date is stored on the card
The machine only has to store information about used cards during their
valid period.

Let's say that your cards are valid for 4 weeks. Let's then say that
you'll never give out more than 2048 cards each week. You encode (and
encrypt, using a symmetric algorithm such as TEA which has a PIC
implementation already) both the date and a number (0-2047) on the card.

You only need 1K of memory to hold all the information. This can be
held on the PIC FLASH. Furthermore, you'll never have to maintain it -
it'll automatically remove entries as needed.

You use one bit per number, and a block of 2048 bits for each week that
the card expires in.

When someone remits their card, look at the date. If it's expired,
reject or eat the card and display "EXPIRED". If it isn't expired, look
into the block that's holding that particular weeks expiration
information and find the bit that goes with that number. If the bit is
set, reject or eat the card "USED". If it isn't set then set it and
dispense the snack.

Once a week, at midnight (or 5 minutes later to be nice) purge one
week's worth of data (2048 bits) and start using that block for
expirations that happen 4 weeks in the future.

Please keep in mind that there are such things as leap days and, if
applicable to your region, daylight saving time. These can be dealt
with in software, but it can be tricky to test completely. You don't
want, for instance, to purge two weeks overnight because the time changed.

This can all be implemented in a single, low end flash PIC. The hand's
off aspect makes it especially nice. No need to track more than 8
thousand cards at once, and no need to store more than a bit each. Of
course the numbers change depending on your application, but the idea is
the same.

I hope this helps.

-Adam

Post by Kresho
Thanks for all the replies.

Post by Russell McMahon
I assume this is for something comparable to a car wash system -
you buy a
"ticket" which is recorded in the system and it is removed once utilised.

Good guess but not quite. What it's for is a snack promotion. People will
receive a card encoded with a number. This number is stored with the
time/date as previously specified. Then the card holder swipes their card on
a vending machine to get their free snack. There will obviously be a limited
time for them to redeem said snack, and that's why the date is required.

Post by Russell McMahon
You didn't mentioned "PIC" one single time, but I'd guess that
that's what you'd like to use, right ? :-)

Yes, or something similar. I'm dabbling with some hitachi H8s at the moment
which might be a better fit for this application.
The b-tree and hashing schemes seem like a possibility. However, I think
Adam's response with his practical analysis is probably fitting here. If the
limited time for redemption is kept short enough I can just go through the
data every now and then and delete expired entries and keep the list lean.
If i come up with another solution i'll keep you posted.
Rgds,
Kresho Sprem
--
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

--
http://www.piclist.com hint: PICList Posts must start with ONE topic:
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

[Kresho]

G'day Adam.

Post by M. Adam Davis
Encode the card number and the date it has to be used by on the card.
You might want to encrypt this information so it can't be easily duplicated.

Can't do. The cards already exist. They are membership cards encoded with a
7 digit number and some other useless (to me) info only. That's why my idea
was to go into a setup mode and swipe the card once to set a token for a
snack, then let the user redeem it by swiping their card at a later time.
The offer may be repeated in the future so it should be a repeatable
process.

Come to think of it, i've found another good reason to use a linear address
space: write endurance. If each card has it's own data address then the
endurance only becomes an issue if a particular user(s) uses their card
endurance times - that's not going to happen if the endurance is 1million
writes.

Nice idea though and very clever approach!

Rgds,

Kresho Sprem

Post by M. Adam Davis
-----Original Message-----
From: pic microcontroller discussion list
Sent: Thursday, 22 April 2004 12:23 PM
Subject: Re: [PIC:] Sorting through millions of numbers
If it's a snack promotion that must be used in a specified amount of

--
http://www.piclist.com hint: PICList Posts must start with ONE topic:
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

[Bob Ammerman]

Ok, lets go back to the basics of the problem:

How many total cards will you system have to handle? I expect this is far
less than the possible 10 million.

Are you concerned about forged cards?

Do only certain members get a snack token? How are they determined?

Are the cards read-only or do they some read-write capability you can use?

How many redemption stations will be needed?

How many transactions per day do you anticipate?

Bob Ammerman
RAm Systems




----- Original Message -----
From: "Kresho" <piclist-xNcZUJDJ+***@public.gmane.org>
To: <PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org>
Sent: Wednesday, April 21, 2004 11:00 PM
Subject: Re: [PIC:] Sorting through millions of numbers

Post by Kresho
G'day Adam.

Post by M. Adam Davis
Encode the card number and the date it has to be used by on the card.
You might want to encrypt this information so it can't be easily duplicated.

Can't do. The cards already exist. They are membership cards encoded with a
7 digit number and some other useless (to me) info only. That's why my idea
was to go into a setup mode and swipe the card once to set a token for a
snack, then let the user redeem it by swiping their card at a later time.
The offer may be repeated in the future so it should be a repeatable
process.
Come to think of it, i've found another good reason to use a linear address
space: write endurance. If each card has it's own data address then the
endurance only becomes an issue if a particular user(s) uses their card
endurance times - that's not going to happen if the endurance is 1million
writes.
Nice idea though and very clever approach!
Rgds,
Kresho Sprem

Post by M. Adam Davis
-----Original Message-----
From: pic microcontroller discussion list
Sent: Thursday, 22 April 2004 12:23 PM
Subject: Re: [PIC:] Sorting through millions of numbers
If it's a snack promotion that must be used in a specified amount of

--
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

--
http://www.piclist.com hint: PICList Posts must start with ONE topic:
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

[Kresho]

Post by Bob Ammerman
How many total cards will you system have to handle? I expect this is far
less than the possible 10 million.

There are up to 10million different card numbers but i can't ever see this
many cards being used. So, for an example, lets say that one person uses the
card every 10 seconds (even that is unlikey) = 8640 cards in a day. Lets
also say that the redemption time is 14 days. Thats 14x8640 = 120960 cards
that could pass through in that time.

Post by Bob Ammerman
Are you concerned about forged cards?

Nope!

Post by Bob Ammerman
Do only certain members get a snack token? How are they determined?

That's up to management. All i told them is they have to swipe the card
(during a setup or recharge mode) to activate a token.

Post by Bob Ammerman
Are the cards read-only or do they some read-write capability you can use?

Read only.

Post by Bob Ammerman
How many redemption stations will be needed?

Not sure as yet, but it may be more than 1. If it's more than 1 then i would
have a single "database board" with all the card usage info and seperate
boards at each machine. The machines would send the card number to the
database board and get the token via some sort of connection yet to be
determined, probably a simple multidrop rs485 thing.

Post by Bob Ammerman
How many transactions per day do you anticipate?

Who knows? I'm taking 1 per 10 seconds as a worst case senario.

Rgds,

Kresho Sprem

Post by Bob Ammerman
-----Original Message-----
From: pic microcontroller discussion list
Sent: Thursday, 22 April 2004 1:28 PM
Subject: Re: [PIC:] Sorting through millions of numbers

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

[Bob Ammerman]

I would suggest a system composed of the following parts:

1: The MASTER station. This would contain a card reader used to set a token
on a card. It would also serve as the database repository, and would drive
an RS485 or similar link to the SLAVE stations to poll for card insertions.
This device would need some form of mass storage, so it could be anything
from a PIC with a bunch of external flash memory to a full-blown PC. Again,
I would use some form of hashtable to store the data, designing it to level
wear on the flash (if flash was used).

2: One or more SLAVE stations, each associated with one snack dispenser.
These would be a simple processor that reads the card, responds to polls and
commands from the MASTER, sends a dispense command to the dispenser, and
lights LEDs or an LCD to indicate things like: 'RETRY CARD', 'NO SNACK
PROGRAMMED', 'EXPIRED', 'DISPENSIING', etc


----- Original Message -----
From: "Kresho" <piclist-xNcZUJDJ+***@public.gmane.org>
To: <PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org>
Sent: Thursday, April 22, 2004 12:02 AM
Subject: Re: [PIC:] Sorting through millions of numbers

Post by Kresho

Post by Bob Ammerman
How many total cards will you system have to handle? I expect this is far
less than the possible 10 million.

There are up to 10million different card numbers but i can't ever see this
many cards being used. So, for an example, lets say that one person uses the
card every 10 seconds (even that is unlikey) = 8640 cards in a day. Lets
also say that the redemption time is 14 days. Thats 14x8640 = 120960 cards
that could pass through in that time.

Post by Bob Ammerman
Are you concerned about forged cards?

Nope!

Post by Bob Ammerman
Do only certain members get a snack token? How are they determined?

That's up to management. All i told them is they have to swipe the card
(during a setup or recharge mode) to activate a token.

Post by Bob Ammerman
Are the cards read-only or do they some read-write capability you can use?

Read only.

Post by Bob Ammerman
How many redemption stations will be needed?

Not sure as yet, but it may be more than 1. If it's more than 1 then i would
have a single "database board" with all the card usage info and seperate
boards at each machine. The machines would send the card number to the
database board and get the token via some sort of connection yet to be
determined, probably a simple multidrop rs485 thing.

Post by Bob Ammerman
How many transactions per day do you anticipate?

Who knows? I'm taking 1 per 10 seconds as a worst case senario.
Rgds,
Kresho Sprem

Post by Bob Ammerman
-----Original Message-----
From: pic microcontroller discussion list
Sent: Thursday, 22 April 2004 1:28 PM
Subject: Re: [PIC:] Sorting through millions of numbers

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

[Alan B. Pearce]

Changed to EE as it doesn't really have any PIC info

Post by M. Adam Davis
When someone remits their card, look at the date. If it's
expired, reject or eat the card and display "EXPIRED".

I think you will find that you always want to eat the card, and never return
it. Any cards left in the hands of the consuming public will be food for the
hackers out there to try and crack your system, by letting them have codes
they can identify and try to match. Sure the same applies to any card that
is distributed, but any unexpired cards will generally have some sort of
attempt at use. You should try and limit the number of cards that a cracker
can get hold of by retaining them in the machine.

The next question becomes how you are going to encode the card. Any sort of
barcode is fair game to some sort of photocopy attempt. You can limit this
by using colours that won't photocopy well, but I suspect that many of the
modern colour "all in one" scanner/printer units would make short work of
that scheme. A possibility may be to have bar codes on both sides, which
need to printed so that the registration between them needs to be reasonably
exact, which should limit the duplication by most photocopy methods, but may
give you hassles in the production of cards.

This then leads you to some form of RFID system as being the least hackable
method. If worked right, it should be possible to make the cards so that
they are reusable after they are reclaimed from the machine, thereby
defraying to some degree the extra cost of the cards.

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

[Jake Anderson]

it sounds like you will have your information loaded every once in a while
from a PC
why not have your pc upload a sorted list then binary search the mmc.

-----Original Message-----
From: pic microcontroller discussion list
[mailto:PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org]On Behalf Of Kresho
Sent: Thursday, April 22, 2004 10:58 AM
To: PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org
Subject: Re: [PIC:] Sorting through millions of numbers


Thanks for all the replies.

Post by Russell McMahon
I assume this is for something comparable to a car wash system -
you buy a
"ticket" which is recorded in the system and it is removed once utilised.

Good guess but not quite. What it's for is a snack promotion. People will
receive a card encoded with a number. This number is stored with the
time/date as previously specified. Then the card holder swipes their card on
a vending machine to get their free snack. There will obviously be a limited
time for them to redeem said snack, and that's why the date is required.

Post by Russell McMahon
You didn't mentioned "PIC" one single time, but I'd guess that
that's what you'd like to use, right ? :-)

Yes, or something similar. I'm dabbling with some hitachi H8s at the moment
which might be a better fit for this application.

The b-tree and hashing schemes seem like a possibility. However, I think
Adam's response with his practical analysis is probably fitting here. If the
limited time for redemption is kept short enough I can just go through the
data every now and then and delete expired entries and keep the list lean.

If i come up with another solution i'll keep you posted.

Rgds,

Kresho Sprem

--
http://www.piclist.com hint: PICList Posts must start with ONE topic:
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

[M. Adam Davis]

This is within the field of Hashing if you expect that you won't ever
need to store all 10,000,000 entries at once. If you don't have a good
upper limit of the maximum entries you need to store, then hashing isn't
a good solution.

Hashing is taking the input, performing some operation on it (usually
taking a larger number and making it smaller) and then outputting the
result. The resulting number, which is smaller, tells you where you can
find that data. So, for instance, if you have 10 million entries, and
you expect that you'll only ever store 100 at a time, then the operation
can be as simple as [card number] mod 100.

This inevitably leads to collisions, where two cards end up with the
same hash number. This is the interesting area, and there are two
well-known ways of overcoming this.

1) Look at the spot the hash points to. If it's empty or the number
matches, great use it. If it's full and the number doesn't match then
simply go along the hash array linearly until you find an empty spot or
the original card number and store it there. The search process is
considerably shortened by this method, even thoug you are still using a
linear search. In this case the hash array has to be as large as the
data you'll need to store - ie, you can run out of space if the array is
too small. Make sure the program accounts for this and either
occasionally deletes old entries, or produces and error message when
approaching a full array. You could also refactor the entire array and
create a larger hashing function, but it'd be difficult to use the card
reader during this time, and testing is more difficult.

2) Each entry in the hash array points to a linked list of data
structures. So hash the card number, go to that spot in the array and
you'll find the biginning pointer to a linked list of entries. Follow
the list until you find the card number, or go to the end and add a new
entry, allocating a new chunk of data in the storage medium. This
overcomes the problem of running out of array space since you can fill
out the entire 10,000,000 entry space if the storage card is large enough.

However, 64MB cards are cheap, and just because a solution isn't 'cool'
doesn't mean it's a bad solution. If I were in your shoes, I'd simply
map a single location to each card (don't need the card number, it's
mapped into the location) and store the 6 byte date/time into the
appropiate spot. It's easier to code, test, and the single fastest
solution.

But if you want an 'elegant' solution, then you can play with hashing.
Note that many memory cards have 512 byte blocks. You can factor this
into the hash algorithm, and place all the cards for a certian hash into
one block. This means that you won't have to read more than one data
block to look up any one card number.

Good luck!

-Adam

Post by Kresho
G'day list.
A machine will have a mag swipe card reader. The cards are encoded with any
number between 0 and 10,000,000 representing an account number. When a card
is swiped i must store the card number, time and date - 9 byte chunks of
data (24bit card number and 6bytes of date/time). When the card is swiped
again i must recall the same information and can then delete it.
Storing heaps of data is no problem - MMC or similar
However, i see an issue in catering for all these cards. Let's say that a
new person swipes his card every 10 seconds. That's 8640 swipes in a day. So
i must store 8640 chunks of data somewhere. The same person may not swipe
his card again for a number of days or weeks. Suddenly, there are thousands
(millions?) of chunks of stored data.
1) Allocate a linearly determined memory address for each particular card.
eg: card #0 data at address h'00', card #1 data at address h'09', card #XYZ
at address 9*XYZ, etc. I would need 10,000,000 x 9 bytes or approx 90mb of
memory. That's do-able with a 128mb MMC, but rather unelegant.
2) I can store the 9byte chunks as i receive them in a free slot in memory.
When the card is swiped again i go through the stored data, from the start
of memory, till i find a match based on the card number, and then i can
access the time/date for that card and delete it leaving a free slot again.
However, if that card is the 1 millionth entry in this memory, it will take
considerable time to find that card number. Reading a million RAM locations
wouldn't be that bad (seconds) but accessing an MMC takes a bit of time and
i can see the time blowing out of all proportions.
Does anyone have any ideas how this could be done in any way other than
option 1 above? My biggest reason for not liking this option is that there
is talk of upscaling the number of possible card numbers to 1x10^12, which
suddenly makes any MMC (or other memory for that matter) look useless.
Rgds,
Kresho Sprem
--
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

--
http://www.piclist.com hint: PICList Posts must start with ONE topic:
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

[Bruce Partridge]

How about a binary tree implemented as a linked list. Or 10 binary trees
based on the first digit of the number.

Of course if the numbers are issued in either sequential order or as a count
of some physical property, the group starting with 1,2,3, and 4 will be more
heavily used than the group starting 5,6,7,8,9

Bruce Partridge
http://www.rebreather.ca

-----Original Message-----
From: pic microcontroller discussion list
Sent: Wednesday, April 21, 2004 9:14 AM
Subject: [PIC:] Sorting through millions of numbers
G'day list.
A machine will have a mag swipe card reader. The cards are
encoded with any
number between 0 and 10,000,000 representing an account number. When a card
is swiped i must store the card number, time and date - 9 byte chunks of
data (24bit card number and 6bytes of date/time). When the card is swiped
again i must recall the same information and can then delete it.

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.659 / Virus Database: 423 - Release Date: 4/15/2004

--
http://www.piclist.com hint: PICList Posts must start with ONE topic:
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

[Jan-Erik Soderholm]

Post by Kresho
A machine will have a mag swipe card reader. The cards are
encoded with any number between 0 and 10,000,000 representing
an account number.

An "account number" as in "bank account number" ? Or is it
some kind us "user account number". Ir simply just a way
to identify each individual card without any other meaning ?

Post by Kresho
When a card is swiped i must store the card
number, time and date - 9 byte chunks of data (24bit card number
and 6bytes of date/time). When the card is swipedagain i must recall
the same information and can then delete it.

You didn't mentioned "PIC" one single time, but I'd guess that
that's what you'd like to use, right ? :-)

What is the importance of this application ?
Checking people passing in to and out of some
nuclear power plant ? Or something "simpler" ?
The actual solution whould probably be *highly*
dependent on this parameter.

What happens if you suddenly lose all data ?
Is that allowed at all ? That's also a very important
design parameter !!


Anyway...

I'd look at some solution where the PIC and card reader
is an I/O device to some computer system running a
decent database to keep track of the persons passing.
Could be UNIX, VMS, whatever.

The PIC doesn't realy store anything, it just passes the
card-number and timestamp over to the main system.
Could use a simple serial (RS232) line.
The main system could be whatever needed to cater for
the security and safety, like having shadowed disks and
so on. Maybe runing an OS that can e clustered to get
true 24*7 uptime, if needed (?).

It would also be rather easy to expand the solution to
multiple registration stations. It doesn't matter if the
person passes "in" and "out" (or whatever they actualy
are doing !) using defferent readers.

Post by Kresho
My biggest reason for not liking this option [PIC/MCC]
is that there is talk of upscaling the number of possible
card numbers to 1x10^12,...

Which definitly tells me that a "real" computer solution probably
is what's needed...

Jan-Erik.

--
http://www.piclist.com hint: PICList Posts must start with ONE topic:
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

[John Ferrell]

Study up on "Sparse Matrix" in whatever Computer Science books you have
available. Most of the really good work in this area happened when storage
was precious, before the mid 1980's.
Depending on your volume you may get to more than one machine. Assuming your
numbers are base 10, your search algorithm should consider that. IF these
numbers are some kind of self check code, you may be able to exclude a great
number of them by simply using a validation routine.

The telephone company used to use a similar system to time long distance
phone calls in the 1960's using a wide Paper tape as storage media. However,
the tape was copied to mag tape on an IBM 7074 system for processing.
John Ferrell
http://DixieNC.US

----- Original Message -----
From: "Kresho" <piclist-xNcZUJDJ+***@public.gmane.org>
To: <PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org>
Sent: Wednesday, April 21, 2004 12:14 PM
Subject: [PIC:] Sorting through millions of numbers

Post by Kresho
G'day list.
A machine will have a mag swipe card reader. The cards are encoded with any
number between 0 and 10,000,000 representing an account number. When a card
is swiped i must store the card number, time and date - 9 byte chunks of
data (24bit card number and 6bytes of date/time). When the card is swiped
again i must recall the same information and can then delete it.
Storing heaps of data is no problem - MMC or similar
However, i see an issue in catering for all these cards. Let's say that a
new person swipes his card every 10 seconds. That's 8640 swipes in a day. So
i must store 8640 chunks of data somewhere. The same person may not swipe
his card again for a number of days or weeks. Suddenly, there are thousands
(millions?) of chunks of stored data.
1) Allocate a linearly determined memory address for each particular card.
eg: card #0 data at address h'00', card #1 data at address h'09', card #XYZ
at address 9*XYZ, etc. I would need 10,000,000 x 9 bytes or approx 90mb of
memory. That's do-able with a 128mb MMC, but rather unelegant.
2) I can store the 9byte chunks as i receive them in a free slot in memory.
When the card is swiped again i go through the stored data, from the start
of memory, till i find a match based on the card number, and then i can
access the time/date for that card and delete it leaving a free slot again.
However, if that card is the 1 millionth entry in this memory, it will take
considerable time to find that card number. Reading a million RAM locations
wouldn't be that bad (seconds) but accessing an MMC takes a bit of time and
i can see the time blowing out of all proportions.
Does anyone have any ideas how this could be done in any way other than
option 1 above? My biggest reason for not liking this option is that there
is talk of upscaling the number of possible card numbers to 1x10^12, which
suddenly makes any MMC (or other memory for that matter) look useless.
Rgds,
Kresho Sprem
--
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

--
http://www.piclist.com hint: PICList Posts must start with ONE topic:
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

[Bob Ammerman]

Think 'hash table'

Bob Ammerman
RAm Systems

----- Original Message -----
From: "Kresho" <piclist-xNcZUJDJ+***@public.gmane.org>
To: <PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org>
Sent: Wednesday, April 21, 2004 12:14 PM
Subject: [PIC:] Sorting through millions of numbers

Post by Kresho
G'day list.
A machine will have a mag swipe card reader. The cards are encoded with any
number between 0 and 10,000,000 representing an account number. When a card
is swiped i must store the card number, time and date - 9 byte chunks of
data (24bit card number and 6bytes of date/time). When the card is swiped
again i must recall the same information and can then delete it.
Storing heaps of data is no problem - MMC or similar
However, i see an issue in catering for all these cards. Let's say that a
new person swipes his card every 10 seconds. That's 8640 swipes in a day. So
i must store 8640 chunks of data somewhere. The same person may not swipe
his card again for a number of days or weeks. Suddenly, there are thousands
(millions?) of chunks of stored data.
1) Allocate a linearly determined memory address for each particular card.
eg: card #0 data at address h'00', card #1 data at address h'09', card #XYZ
at address 9*XYZ, etc. I would need 10,000,000 x 9 bytes or approx 90mb of
memory. That's do-able with a 128mb MMC, but rather unelegant.
2) I can store the 9byte chunks as i receive them in a free slot in memory.
When the card is swiped again i go through the stored data, from the start
of memory, till i find a match based on the card number, and then i can
access the time/date for that card and delete it leaving a free slot again.
However, if that card is the 1 millionth entry in this memory, it will take
considerable time to find that card number. Reading a million RAM locations
wouldn't be that bad (seconds) but accessing an MMC takes a bit of time and
i can see the time blowing out of all proportions.
Does anyone have any ideas how this could be done in any way other than
option 1 above? My biggest reason for not liking this option is that there
is talk of upscaling the number of possible card numbers to 1x10^12, which
suddenly makes any MMC (or other memory for that matter) look useless.
Rgds,
Kresho Sprem
--
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

--
http://www.piclist.com hint: PICList Posts must start with ONE topic:
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

[michael brown]

Post by Bob Ammerman
Think 'hash table'

I was thinking b-tree.

michael brown

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

[Bob Ammerman]

----- Original Message -----
From: "michael brown" <spam-me-OvDHpeKI7IRFrtkqe/***@public.gmane.org>
To: <PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org>
Sent: Wednesday, April 21, 2004 6:17 PM
Subject: Re: [PIC:] Sorting through millions of numbers

Post by michael brown

Post by Bob Ammerman
Think 'hash table'

I was thinking b-tree.
michael brown

Generally 'hash table' is better than 'b-tree' unless you have a need for
in-order traversal of the data. By 'better', I mean that coding efforts are
_much_ simpler, and inserts and finds are faster. However, the 'b-tree' is
typically better at storage usage. That would probably not matter in this
application, since you'd need enough storage to hold the worst case data
anyway.

Bob Ammerman

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

[Ben Hencke]

Post by michael brown

Post by Bob Ammerman
Think 'hash table'

I was thinking b-tree.
michael brown

I would really reccomend that you stay away from hashing at all. Never
hash a serial number that already fits into a reasonable ammount of
space. Hashing is really only useful when trying to take a large key of
data (ie a string, file, whole record, etc) and make it into a new key
that can be easily sorted/indexed/etc. A card # that is 1 to 10 milion
already fits nicely into 24 bits. Hashing this small ammount of data
will only complicate things.

A b-tree structure would be optimal for speed, but with the small
ammount of records you will be dealing with, a keyed linked list (aka
dictionary) will be far easier to code on a PIC.

Start by breaking your data storage (flash, battery backed sram, hdd,
etc) into blocks/records that are big enough to hold the data you need.
That way you can index each record. In this case the card#, ptr to next
record in list, date activated, data, and anything else you plan to
store.

When you insert a record (by scanning the card) traverse the list until
you find the 2 records that your new record will need to insert itself
between. That should be as simple as a less than check on the card #s.
You can even speed that up by comparing it byte by byte (msb first).
Then allocate a new record. You can keep a counter and incrementally
scan the memory for free/expired records to average the wear on the
flash. Make the new record point to the next in list and make the
previous in list point to the new record.

When you need to redeam a card, just run through the list. When you
find it, make the previous record point to the record after the
redeamed record and clear the redeamed record. double check the date
only if you want minute by minute expiration.

Every midnight scan the list and delete (as described above) records
that expire.

Each time you insert or remove a record you will end up altering 3
records. If you have 8640 activations and 8640 redemptions/expirations
a day thats 51,600 distributed flash/eeprom record writes per day. If
your flash was just big enoug to hold 120960 records, thats average of
about half a write per day. The larger a flash you use, the more
distributed the wear will be. If it is a heavily used vending machine
in a long term installation you might want to use battery backed sram.
Maxim has good lithium backed srams with an integrated RTC that lasts
10 years without power in a nice dip package. They are pricy for sram,
but since you share the data source it might be worth it in terms of
design complexity and service calls.


Hope that helps,
Ben Hencke









__________________________________
Do you Yahoo!?
Yahoo! Photos: High-quality 4x6 digital prints for 25"
http://photos.yahoo.com/ph/print_splash

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

[M. Adam Davis]

While it's fairly acedemic, I'm interested in your thoughts one why
you'd choose a btree over a hash table.

If, as I am assuming, these are ID cards that are dispensed
sequentually, then at any given time the btree will be unevenly
weighted. This could lead to very deep, sparse trees that could require
more storage space than a simple linear array.

Of course, this may be a limitation to my understanding of a btree -
which is why I'm asking. I've only implemented very simple trees or
very complex self-balancing trees. The self balancing method I used,
however, would not be well suited to flash storage since more than one
element in the tree might change for each newly inserted element, and
the simple tree did not attempt to maintain any balance since there were
no memory bounds in that project.

-Adam

Post by michael brown

Post by Bob Ammerman
Think 'hash table'

I was thinking b-tree.
michael brown
--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

[Wouter van Ooijen]

Post by M. Adam Davis
Of course, this may be a limitation to my understanding of a btree -
which is why I'm asking. I've only implemented very simple trees or
very complex self-balancing trees. The self balancing method I used,
however, would not be well suited to flash storage since more than one
element in the tree might change for each newly inserted element, and
the simple tree did not attempt to maintain any balance since
there were
no memory bounds in that project.

I implemented a b+ tree once (in my first job fresh from university).
IIRC the algorithm certainly allows for an insert to force updates of
lots of nodes, but there is a statistical average that is much lower.
But sequential inserting might defeat this (the statistics are based on
randomness). IIRC I did something to overcome exactly this.

But as said, when sorted traversal is not needed a hash table is the way
to go.

Wouter van Ooijen

-- -------------------------------------------
Van Ooijen Technische Informatica: www.voti.nl
consultancy, development, PICmicro products

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

[M. Adam Davis]

Having thought about it a little further, I suspect your project will
benefit from a practical analysis.

First of all, define how much data you need to store and how long you
have to store it before you can clear it. I doubt you need to store
information that's over a year old, for instance. Chances are good that
this type of project requires the data to be offloaded on a fiarly
regular basis, or moderately old data is useless (ie, a simple building
security system may only need data going back 2 weeks).

Even if you had a person standing at the reader with all 10,000,000
cards swiping a new card once every second until they exhausted all 10
million cards it would take 4 months to go through all those cards.

If you have more than 1,000 people a day swiping their card then data
storage and lookup is not the worst problem you'll run into before the
project is done.

So try and come up with the maximum number of uses per day, then the
maximum days between offloading (or maximum needed backlog, etc) and
that will tell you how many memory slots you'll need. It won't matter
how many cards exist, only how many cards will be swiped at that
location between clean ups and off loads.

If you need to keep data indefinitely, then you must allocate space for
each possible card number, and if you're doing that then there's /no/
reason to use a special storage mechanism. If you must accept 1
trillion card numbers then I'd strongly suggest looking into compression
and using a real computer system (terabyte raid array) because nothing
else at this point in time is going to hold the information you want for
that many cards 'forever'.

-Adam

Post by Kresho
G'day list.
A machine will have a mag swipe card reader. The cards are encoded with any
number between 0 and 10,000,000 representing an account number. When a card
is swiped i must store the card number, time and date - 9 byte chunks of
data (24bit card number and 6bytes of date/time). When the card is swiped
again i must recall the same information and can then delete it.
Storing heaps of data is no problem - MMC or similar
However, i see an issue in catering for all these cards. Let's say that a
new person swipes his card every 10 seconds. That's 8640 swipes in a day. So
i must store 8640 chunks of data somewhere. The same person may not swipe
his card again for a number of days or weeks. Suddenly, there are thousands
(millions?) of chunks of stored data.
1) Allocate a linearly determined memory address for each particular card.
eg: card #0 data at address h'00', card #1 data at address h'09', card #XYZ
at address 9*XYZ, etc. I would need 10,000,000 x 9 bytes or approx 90mb of
memory. That's do-able with a 128mb MMC, but rather unelegant.
2) I can store the 9byte chunks as i receive them in a free slot in memory.
When the card is swiped again i go through the stored data, from the start
of memory, till i find a match based on the card number, and then i can
access the time/date for that card and delete it leaving a free slot again.
However, if that card is the 1 millionth entry in this memory, it will take
considerable time to find that card number. Reading a million RAM locations
wouldn't be that bad (seconds) but accessing an MMC takes a bit of time and
i can see the time blowing out of all proportions.
Does anyone have any ideas how this could be done in any way other than
option 1 above? My biggest reason for not liking this option is that there
is talk of upscaling the number of possible card numbers to 1x10^12, which
suddenly makes any MMC (or other memory for that matter) look useless.
Rgds,
Kresho Sprem
--
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

--
http://www.piclist.com hint: PICList Posts must start with ONE topic:
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

[Russell McMahon]

Post by Kresho
A machine will have a mag swipe card reader. The cards are encoded with any
number between 0 and 10,000,000 representing an account number. When a card
is swiped i must store the card number, time and date - 9 byte chunks of
data (24bit card number and 6bytes of date/time). When the card is swiped
again i must recall the same information and can then delete it.

Answers so far sound good to me - hashing, B Tree, brute force memory
locations and more.
I assume this is for something comparable to a car wash system - you buy a
"ticket" which is recorded in the system and it is removed once utilised.

I'd like to address another issue. You need to be really really really
confident of your data integrity here. With such a large number of potential
cards , if it is possible for one card to be read wrongly and thereby return
a valid code from another card, you are in deep trouble. Depending on what
the use of the card is this could be just customer annoyance (car washes) or
substantial expense (trying to find out where Elvis is inside the secure
area when he's already left the building.) You may not have control over the
encoding, but if you do, you want as much data integrity as you can get.
10,000,000 potential numbers is binary 24 bits. You want to add at least a
32 bit CRC to that, and possibly forward error correcting.


RM

--
http://www.piclist.com hint: PICList Posts must start with ONE topic:
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

[Bob Axtell]

Looks like the car wash enable situation all right.

I would save the 24 bits (3 bytes) + 2-byte "minutes-per-day" + 2-byte
"day-of-year" field. That's 7 bytes with a coupla bits unused for flags
such as "discard after 24hrs" or "discard after 30 days", etc.

I'd organize the field so that the DOY is first, followed by MOD. That
way, discards more easily performed than with random searches.

--Bob

Post by Russell McMahon

Post by Kresho
A machine will have a mag swipe card reader. The cards are encoded with

any

Post by Kresho
number between 0 and 10,000,000 representing an account number. When a

card

Post by Kresho
is swiped i must store the card number, time and date - 9 byte chunks of
data (24bit card number and 6bytes of date/time). When the card is swiped
again i must recall the same information and can then delete it.

Answers so far sound good to me - hashing, B Tree, brute force memory
locations and more.
I assume this is for something comparable to a car wash system - you buy a
"ticket" which is recorded in the system and it is removed once utilised.
I'd like to address another issue. You need to be really really really
confident of your data integrity here. With such a large number of potential
cards , if it is possible for one card to be read wrongly and thereby return
a valid code from another card, you are in deep trouble. Depending on what
the use of the card is this could be just customer annoyance (car washes) or
substantial expense (trying to find out where Elvis is inside the secure
area when he's already left the building.) You may not have control over the
encoding, but if you do, you want as much data integrity as you can get.
10,000,000 potential numbers is binary 24 bits. You want to add at least a
32 bit CRC to that, and possibly forward error correcting.
RM

--

Replies: NOTE-Script, EXE,BAT and COM
files will be rejected by server
--------------
Bob Axtell
PIC Hardware & Firmware Dev
http://beam.to/baxtell
1-520-219-2363

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

[Tom]

Ken Pergola (and maybe a few others) mentioned recently about using a
Cybiko Classic hand-held computer for a cheap and simple serial terminal.

Question for anyone who knows about Cybikos: what cable can be used with a
Cybiko Xtreme for serial data? Or is it possible?

Thanks!
Tom

--
http://www.piclist.com hint: PICList Posts must start with ONE topic:
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

[Kevin Olalde]

none, I think.

The xtreme only has a usb port, and the sdk never included the needed
documentation to program to it. Get a classic, they're a bit cheaper too.

Kevin

Post by Tom
Ken Pergola (and maybe a few others) mentioned recently about using a
Cybiko Classic hand-held computer for a cheap and simple serial terminal.
Question for anyone who knows about Cybikos: what cable can be used with a
Cybiko Xtreme for serial data? Or is it possible?
Thanks!
Tom

--
http://www.piclist.com hint: PICList Posts must start with ONE topic:
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

[Sergio Masci]

----- Original Message -----
From: Kevin Olalde <kevin-***@public.gmane.org>
To: <PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org>
Sent: Thursday, April 22, 2004 2:00 AM
Subject: Re: [OT:] Cybiko serial cable

Post by Kevin Olalde
none, I think.
The xtreme only has a usb port, and the sdk never included the needed
documentation to program to it. Get a classic, they're a bit cheaper too.
Kevin

Also xtremes seem to be less reliable than the classics. I bought a couple of
classics about a year ago to experiment with. My son regularly uses one to play
MP3s on. Two of his friends decided to buy extremes. These have both had to be
replaced after developing faults.

Regards
Sergio Masci

--
http://www.piclist.com hint: PICList Posts must start with ONE topic:
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

[John Ferrell]

Classics are currently $14.99US at Toysrus.com as a clearence item.

I ordered a few out of curiousity...
John Ferrell
http://DixieNC.US

----- Original Message -----
From: "Sergio Masci" <smpl-1uslQqWFlHF+cjeuK/***@public.gmane.org>
To: <PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org>
Sent: Wednesday, April 21, 2004 9:56 PM
Subject: Re: [OT:] Cybiko serial cable

Post by Sergio Masci
Also xtremes seem to be less reliable than the classics. I bought a couple of
classics about a year ago to experiment with. My son regularly uses one to play
MP3s on. Two of his friends decided to buy extremes. These have both had to be
replaced after developing faults.
Regards
Sergio Masci
--
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

[Dominic Stratten]

There seems to be a lot of old surplus stock out there. I bought 12 of them
and 6 had corroded battery contacts and dead prismatic cells.

Managed to get a few more working with some cell swapping though but be
warned - be prepared for a few failures.

Dom

-----Original Message-----
From: pic microcontroller discussion list
[mailto:PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org]On Behalf Of John Ferrell
Sent: 22 April 2004 15:02
To: PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org
Subject: Re: [OT:] Cybiko serial cable


Classics are currently $14.99US at Toysrus.com as a clearence item.

I ordered a few out of curiousity...
John Ferrell
http://DixieNC.US

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

[Tom]

Thanks to all who responded!

As it turns out, I already had two Xtreme units; I was wondering how to get
serial data out of it other than using the usb port.

I took Johns advice and ordered 2 classics from toysrus - 15 clams is a
pretty low gamble!

We continually seem to have gadgets around the shop that spit out 232 data
and I plan on using these to be able to quickly check them out, go/no-go
fashion.

Tom

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

[John Ferrell]

Now all I have to do is find some time to work with them.
I have two AVR Butterflies that I bought on my desk already...Sure is a lot
for $20!

John Ferrell
http://DixieNC.US

----- Original Message -----
From: "Tom" <krist-***@public.gmane.org>
To: <PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org>
Sent: Thursday, April 22, 2004 1:17 PM
Subject: Re: [OT:] Cybiko serial cable

Post by Tom
Thanks to all who responded!
As it turns out, I already had two Xtreme units; I was wondering how to get
serial data out of it other than using the usb port.
I took Johns advice and ordered 2 classics from toysrus - 15 clams is a
pretty low gamble!
We continually seem to have gadgets around the shop that spit out 232 data
and I plan on using these to be able to quickly check them out, go/no-go
fashion.
Tom
--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

[Randy Glenn]

Anyone know if they're in-store at that price? I'm in Canada, so I can't
check, but I'll make the trip if they are.

-Randy Glenn

Post by Dominic Stratten
There seems to be a lot of old surplus stock out there. I bought 12 of them
and 6 had corroded battery contacts and dead prismatic cells.
Managed to get a few more working with some cell swapping though but be
warned - be prepared for a few failures.
Dom
-----Original Message-----
From: pic microcontroller discussion list
Sent: 22 April 2004 15:02
Subject: Re: [OT:] Cybiko serial cable
Classics are currently $14.99US at Toysrus.com as a clearence item.
I ordered a few out of curiousity...
John Ferrell
http://DixieNC.US
--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

[John Ferrell]

They are $9.95 now...
I got my order. The two I opened seem pretty neat. 900mhz communications,
tiny keyboard, LCD.
I am not sure all of the software is still available for the Classics
though, I am still looking. I may have to start begging!

It is definately worth the close out prices though.

John Ferrell
http://DixieNC.US

----- Original Message -----
From: "John Ferrell" <johnferrell-6GKhM/KhJe7wFRIe+***@public.gmane.org>
To: <PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org>
Sent: Thursday, April 22, 2004 10:02 AM
Subject: Re: [OT:] Cybiko serial cable

Post by John Ferrell
Classics are currently $14.99US at Toysrus.com as a clearence item.
I ordered a few out of curiousity...
John Ferrell
http://DixieNC.US
----- Original Message -----
Sent: Wednesday, April 21, 2004 9:56 PM
Subject: Re: [OT:] Cybiko serial cable

Post by Sergio Masci
Also xtremes seem to be less reliable than the classics. I bought a

couple

Post by John Ferrell
of

Post by Sergio Masci
classics about a year ago to experiment with. My son regularly uses one

to

Post by John Ferrell
play

Post by Sergio Masci
MP3s on. Two of his friends decided to buy extremes. These have both had

to be

Post by Sergio Masci
replaced after developing faults.
Regards
Sergio Masci
--
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

--
http://www.piclist.com#nomail Going offline? Don't AutoReply us!
email listserv-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org with SET PICList DIGEST in the body

[Shawn Wilton]

Where are they on closeout?

Post by John Ferrell
They are $9.95 now...
I got my order. The two I opened seem pretty neat. 900mhz communications,
tiny keyboard, LCD.
I am not sure all of the software is still available for the Classics
though, I am still looking. I may have to start begging!
It is definately worth the close out prices though.
John Ferrell
http://DixieNC.US
----- Original Message -----
Sent: Thursday, April 22, 2004 10:02 AM
Subject: Re: [OT:] Cybiko serial cable

Post by John Ferrell
Classics are currently $14.99US at Toysrus.com as a clearence item.
I ordered a few out of curiousity...
John Ferrell
http://DixieNC.US
----- Original Message -----
Sent: Wednesday, April 21, 2004 9:56 PM
Subject: Re: [OT:] Cybiko serial cable

Post by Sergio Masci
Also xtremes seem to be less reliable than the classics. I bought a

couple

Post by John Ferrell
of

Post by Sergio Masci
classics about a year ago to experiment with. My son regularly uses one

to

Post by John Ferrell
play

Post by Sergio Masci
MP3s on. Two of his friends decided to buy extremes. These have both had

to be

Post by Sergio Masci
replaced after developing faults.
Regards
Sergio Masci
--
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

--
http://www.piclist.com#nomail Going offline? Don't AutoReply us!

--
http://www.piclist.com#nomail Going offline? Don't AutoReply us!
email listserv-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org with SET PICList DIGEST in the body

[John Ferrell]

ToysRus.com, direct or through Amazon.com

John Ferrell
http://DixieNC.US

----- Original Message -----
From: "Shawn Wilton" <shawn-***@public.gmane.org>
To: <PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org>
Sent: Thursday, April 29, 2004 7:32 PM
Subject: Re: [OT:] Cybiko serial cable

Post by Shawn Wilton
Where are they on closeout?

Post by John Ferrell
They are $9.95 now...
I got my order. The two I opened seem pretty neat. 900mhz

communications,

Post by Shawn Wilton

Post by John Ferrell
tiny keyboard, LCD.
I am not sure all of the software is still available for the Classics
though, I am still looking. I may have to start begging!
It is definately worth the close out prices though.
John Ferrell
http://DixieNC.US
----- Original Message -----
Sent: Thursday, April 22, 2004 10:02 AM
Subject: Re: [OT:] Cybiko serial cable

Post by John Ferrell
Classics are currently $14.99US at Toysrus.com as a clearence item.
I ordered a few out of curiousity...
John Ferrell
http://DixieNC.US
----- Original Message -----
Sent: Wednesday, April 21, 2004 9:56 PM
Subject: Re: [OT:] Cybiko serial cable

Post by Sergio Masci
Also xtremes seem to be less reliable than the classics. I bought a

couple

Post by John Ferrell
of

Post by Sergio Masci
classics about a year ago to experiment with. My son regularly uses one

to

Post by John Ferrell
play

Post by Sergio Masci
MP3s on. Two of his friends decided to buy extremes. These have both had

to be

Post by Sergio Masci
replaced after developing faults.
Regards
Sergio Masci
--
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

--
http://www.piclist.com#nomail Going offline? Don't AutoReply us!

--
http://www.piclist.com#nomail Going offline? Don't AutoReply us!

--
http://www.piclist.com#nomail Going offline? Don't AutoReply us!
email listserv-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org with SET PICList DIGEST in the body

[Shawn Wilton]

Cool, thx.

Post by John Ferrell
ToysRus.com, direct or through Amazon.com
John Ferrell
http://DixieNC.US
----- Original Message -----
Sent: Thursday, April 29, 2004 7:32 PM
Subject: Re: [OT:] Cybiko serial cable

Post by Shawn Wilton
Where are they on closeout?

Post by John Ferrell
They are $9.95 now...
I got my order. The two I opened seem pretty neat. 900mhz

communications,

Post by Shawn Wilton

Post by John Ferrell
tiny keyboard, LCD.
I am not sure all of the software is still available for the Classics
though, I am still looking. I may have to start begging!
It is definately worth the close out prices though.
John Ferrell
http://DixieNC.US
----- Original Message -----
Sent: Thursday, April 22, 2004 10:02 AM
Subject: Re: [OT:] Cybiko serial cable

Post by John Ferrell
Classics are currently $14.99US at Toysrus.com as a clearence item.
I ordered a few out of curiousity...
John Ferrell
http://DixieNC.US
----- Original Message -----
Sent: Wednesday, April 21, 2004 9:56 PM
Subject: Re: [OT:] Cybiko serial cable

Post by Sergio Masci
Also xtremes seem to be less reliable than the classics. I bought a

couple

Post by John Ferrell
of

Post by Sergio Masci
classics about a year ago to experiment with. My son regularly uses one

to

Post by John Ferrell
play

Post by Sergio Masci
MP3s on. Two of his friends decided to buy extremes. These have both

had

Post by Shawn Wilton

Post by John Ferrell

Post by John Ferrell
to be

Post by Sergio Masci
replaced after developing faults.
Regards
Sergio Masci
--
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

--
http://www.piclist.com#nomail Going offline? Don't AutoReply us!

--
http://www.piclist.com#nomail Going offline? Don't AutoReply us!

--
http://www.piclist.com#nomail Going offline? Don't AutoReply us!

--
http://www.piclist.com#nomail Going offline? Don't AutoReply us!
email listserv-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org with SET PICList DIGEST in the body

[redtock8]

Michael Olsen,
I get this warning from your e-mail.


Security Warning

There are security problems with this message.
Please review the highlighted items listed below:

Message has been tampered with
You have not yet made a decision about whether to trust the digital ID
used to sign this message
The digital ID has not expired
The sender and digital ID have the same e-mail address
The digital ID has not been revoked or revocation information for this
certificate could not be determined.
There are no other problems with the digital ID

--
http://www.piclist.com#nomail Going offline? Don't AutoReply us!
email listserv-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org with SET PICList DIGEST in the body

[Michael Olson]

My appologies, I had thunderbird set up to automatically sign everything
going out and thats probably not terribly appropriate for a public
mailing list. I'll make sure that doesn't happen again.

-- Mike Olson

Post by redtock8
Michael Olsen,
I get this warning from your e-mail.
Security Warning
There are security problems with this message.
Message has been tampered with
You have not yet made a decision about whether to trust the digital ID
used to sign this message
The digital ID has not expired
The sender and digital ID have the same e-mail address
The digital ID has not been revoked or revocation information for this
certificate could not be determined.
There are no other problems with the digital ID
--
http://www.piclist.com#nomail Going offline? Don't AutoReply us!

--
http://www.piclist.com#nomail Going offline? Don't AutoReply us!
email listserv-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org with SET PICList DIGEST in the body

[John Pearson]

Why are these devices on close-out? Are they going out of business or
perhaps new models are arriving?

John
----- Original Message -----
From: "Michael Olson" <olson-1Q7CFEBLPbrltQq4i4S/***@public.gmane.org>
To: <PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org>
Sent: Thursday, April 29, 2004 5:05 PM
Subject: Re: [OT:] Cybiko serial cable

Post by Michael Olson
My appologies, I had thunderbird set up to automatically sign everything
going out and thats probably not terribly appropriate for a public
mailing list. I'll make sure that doesn't happen again.
-- Mike Olson

Post by redtock8
Michael Olsen,
I get this warning from your e-mail.
Security Warning
There are security problems with this message.
Message has been tampered with
You have not yet made a decision about whether to trust the digital ID
used to sign this message
The digital ID has not expired
The sender and digital ID have the same e-mail address
The digital ID has not been revoked or revocation information for this
certificate could not be determined.
There are no other problems with the digital ID
--
http://www.piclist.com#nomail Going offline? Don't AutoReply us!

--
http://www.piclist.com#nomail Going offline? Don't AutoReply us!

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[Herbert Graf]

Post by John Pearson
Why are these devices on close-out? Are they going out of business or
perhaps new models are arriving?

Anybody know of a Canadian source for these things? Thanks, TTYL

----------------------------------
Herbert's PIC Stuff:
http://repatch.dyndns.org:8383/pic_stuff/

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[Jake Anderson]

anybody know an Australian source?

-----Original Message-----
From: pic microcontroller discussion list
[mailto:PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org]On Behalf Of Herbert Graf
Sent: Friday, April 30, 2004 2:58 PM
To: PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org
Subject: Re: [OT:] Cybiko serial cable

Post by John Pearson
Why are these devices on close-out? Are they going out of business or
perhaps new models are arriving?

Anybody know of a Canadian source for these things? Thanks, TTYL

----------------------------------
Herbert's PIC Stuff:
http://repatch.dyndns.org:8383/pic_stuff/

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[Denny Esterline]

Cybiko classics are about 4-6 years ago, Cybiko Extreme (USB only no serial
port, not as usefull for our purposes) replaced them 2 or 3 years ago. I
suspect Extremes will be on clearance soon enough.

I bought the last two at my local Media Play store a couple years ago for
$14 ea. I had plans on using them as a cheap wireless serial connection,
but they found a home in my closet and haven't moved since.

-Denny

Post by John Pearson
Why are these devices on close-out? Are they going out of business or
perhaps new models are arriving?
John

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[John Ferrell]

New model with USB interface instead of RS232.
The older model is probably easier for Pic interfacing.
Besides the the new model is about $100.

John Ferrell
http://DixieNC.US

----- Original Message -----
From: "John Pearson" <xero-***@public.gmane.org>
To: <PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org>
Sent: Friday, April 30, 2004 12:47 AM
Subject: Re: [OT:] Cybiko serial cable

Post by John Pearson
Why are these devices on close-out? Are they going out of business or
perhaps new models are arriving?
John
----- Original Message -----
Sent: Thursday, April 29, 2004 5:05 PM
Subject: Re: [OT:] Cybiko serial cable

Post by Michael Olson
My appologies, I had thunderbird set up to automatically sign everything
going out and thats probably not terribly appropriate for a public
mailing list. I'll make sure that doesn't happen again.
-- Mike Olson

Post by redtock8
Michael Olsen,
I get this warning from your e-mail.
Security Warning
There are security problems with this message.
Message has been tampered with
You have not yet made a decision about whether to trust the

digital

Post by John Pearson
ID

Post by Michael Olson

Post by redtock8
used to sign this message
The digital ID has not expired
The sender and digital ID have the same e-mail address
The digital ID has not been revoked or revocation information for

this

Post by Michael Olson

Post by redtock8
certificate could not be determined.
There are no other problems with the digital ID
--
http://www.piclist.com#nomail Going offline? Don't AutoReply us!

--
http://www.piclist.com#nomail Going offline? Don't AutoReply us!

--
http://www.piclist.com hint: To leave the PICList

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[William Bross]

Can anybody who just purchased any classics tell us if the serial cable
comes with the unit and if not, what type connector does the unit need?

Bill

-----Original Message-----
From: pic microcontroller discussion list
[mailto:PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org] On Behalf Of John Ferrell
Sent: Thursday, April 29, 2004 7:37 PM
To: PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org
Subject: Re: [OT:] Cybiko serial cable


ToysRus.com, direct or through Amazon.com

John Ferrell
http://DixieNC.US

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[Tom]

Post by William Bross
Can anybody who just purchased any classics tell us if the serial cable
comes with the unit and if not, what type connector does the unit need?
Bill

The classic model does come with a serial cable. It has some sort of
proprietary looking connector on the Cybiko end and a 9 pin d-sub on the
other end.
Tom

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[William Bross]

Thanks for the info. Makes the ten buck deal even sweeter! I'll be
able to nail one to the wall and save some bench space. It should be
easier than firing up the computer just to check a datastream for
activity.

Bill

-----Original Message-----
From: pic microcontroller discussion list
[mailto:PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org] On Behalf Of Tom
Sent: Friday, April 30, 2004 11:55 AM
To: PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org
Subject: Re: [OT:] Cybiko serial cable

Post by William Bross
Can anybody who just purchased any classics tell us if the serial cable
comes with the unit and if not, what type connector does the unit need?
Bill

The classic model does come with a serial cable. It has some sort of
proprietary looking connector on the Cybiko end and a 9 pin d-sub on the
other end. Tom

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[Jake Anderson]

anybody in the USA interested in picking some of these up and shipping them
to the rest of us scum? (ie people outside america) ;->
Promise we wont use them to simulate nuclear blasts or nothin.

what is it with america and not shipping outside USA? seems in every other
country if you arent willing to ship international your a nutter.

-----Original Message-----
From: pic microcontroller discussion list
[mailto:PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org]On Behalf Of William Bross
Sent: Saturday, May 01, 2004 2:02 AM
To: PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org
Subject: Re: [OT:] Cybiko serial cable


Thanks for the info. Makes the ten buck deal even sweeter! I'll be
able to nail one to the wall and save some bench space. It should be
easier than firing up the computer just to check a datastream for
activity.

Bill

-----Original Message-----
From: pic microcontroller discussion list
[mailto:PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org] On Behalf Of Tom
Sent: Friday, April 30, 2004 11:55 AM
To: PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org
Subject: Re: [OT:] Cybiko serial cable

Post by William Bross
Can anybody who just purchased any classics tell us if the serial cable
comes with the unit and if not, what type connector does the unit need?
Bill

The classic model does come with a serial cable. It has some sort of
proprietary looking connector on the Cybiko end and a 9 pin d-sub on the
other end. Tom

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[Shawn Wilton]

It's because we have all the cool tech and don't want to share. But
rest assured, if you have something we want, we'll just coerce your
scientists to defect. :-P

Post by Jake Anderson
anybody in the USA interested in picking some of these up and shipping them
to the rest of us scum? (ie people outside america) ;->
Promise we wont use them to simulate nuclear blasts or nothin.
what is it with america and not shipping outside USA? seems in every other
country if you arent willing to ship international your a nutter.
-----Original Message-----
From: pic microcontroller discussion list
Sent: Saturday, May 01, 2004 2:02 AM
Subject: Re: [OT:] Cybiko serial cable
Thanks for the info. Makes the ten buck deal even sweeter! I'll be
able to nail one to the wall and save some bench space. It should be
easier than firing up the computer just to check a datastream for
activity.
Bill
-----Original Message-----
From: pic microcontroller discussion list
Sent: Friday, April 30, 2004 11:55 AM
Subject: Re: [OT:] Cybiko serial cable

Post by William Bross
Can anybody who just purchased any classics tell us if the serial cable
comes with the unit and if not, what type connector does the unit need?
Bill

The classic model does come with a serial cable. It has some sort of
proprietary looking connector on the Cybiko end and a 9 pin d-sub on the
other end. Tom
--
http://www.piclist.com hint: To leave the PICList
--
http://www.piclist.com hint: To leave the PICList
--
http://www.piclist.com hint: To leave the PICList

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[Dominic Stratten]

The biggest problem with these is the packaging. It accounts for around 70%
of the total weight of the units so sending them "abroad" makes them a
little uneconomical.

The other problem with these is that they used Prismatic NiMh batteries.
These cost around 10ukp here in the Uk - about what I can buy a Cybiko
classic for.

I bought a crate of these to sell them on and had a failure rate on the
batteries out of the boxes of around 50%. With a bit of contact cleaning and
binning dead cells, I got 75% of the units working.

Chances are that if someone buys a bulk lot of them for you, you'll have the
same problems. The units will not work properly with dead cells (even on the
wall warts) and can do some strange things when the cells are starting to
die.

Just a quick warning - they are great little units otherwise.

Dom

-----Original Message-----
From: pic microcontroller discussion list
[mailto:PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org]On Behalf Of Jake Anderson
Sent: 30 April 2004 17:22
To: PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org
Subject: Re: [OT:] Cybiko serial cable


anybody in the USA interested in picking some of these up and shipping them
to the rest of us scum? (ie people outside america) ;->
Promise we wont use them to simulate nuclear blasts or nothin.

what is it with america and not shipping outside USA? seems in every other
country if you arent willing to ship international your a nutter.

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[John Ferrell]

Perhaps I am too optimistc but I plan to replace any NiMh cells with more
readily obtainable ones.
John Ferrell
http://DixieNC.US

----- Original Message -----
From: "Dominic Stratten" <dominic.stratten-1uslQqWFlHF+cjeuK/***@public.gmane.org>
To: <PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org>
Sent: Friday, April 30, 2004 12:31 PM
Subject: Re: [OT:] Cybiko serial cable

Post by Dominic Stratten
The biggest problem with these is the packaging. It accounts for around 70%
of the total weight of the units so sending them "abroad" makes them a
little uneconomical.
The other problem with these is that they used Prismatic NiMh batteries.
These cost around 10ukp here in the Uk - about what I can buy a Cybiko
classic for.
I bought a crate of these to sell them on and had a failure rate on the
batteries out of the boxes of around 50%. With a bit of contact cleaning and
binning dead cells, I got 75% of the units working.
Chances are that if someone buys a bulk lot of them for you, you'll have the
same problems. The units will not work properly with dead cells (even on the
wall warts) and can do some strange things when the cells are starting to
die.
Just a quick warning - they are great little units otherwise.
Dom
-----Original Message-----
From: pic microcontroller discussion list
Sent: 30 April 2004 17:22
Subject: Re: [OT:] Cybiko serial cable
anybody in the USA interested in picking some of these up and shipping them
to the rest of us scum? (ie people outside america) ;->
Promise we wont use them to simulate nuclear blasts or nothin.
what is it with america and not shipping outside USA? seems in every other
country if you arent willing to ship international your a nutter.
--
http://www.piclist.com hint: To leave the PICList

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[Ken Pergola]

I was hoping someone could educate me on something that I see and hear a lot
of comments about. I hear a lot of claims that Linux is more secure than
Windows, but I don't usually see people elaborate on why this claim is made
after they make it.

Isn't C++ the underlying language that both operating systems are written
in?

And if so, wouldn't both operating systems be *equal* in their
susceptibility to buffer overrun attacks, for example, due to the pitfalls
of C-string functions like 'strcpy', 'strcat' in C++?

Perhaps this is narrowing things down to such a simplistic and specific
example, but I feel it is a fair question to ask. Thank you for sharing any
comments on this.

Best regards,

Ken Pergola

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[Shawn Wilton]

For the most part, security is a matter of the programmer writing the
code. If the programmer knows what they're doing, then all is well. If
they don't know how to write secure code, then you get in to problem areas.

-Shawn

Post by Ken Pergola
I was hoping someone could educate me on something that I see and hear a lot
of comments about. I hear a lot of claims that Linux is more secure than
Windows, but I don't usually see people elaborate on why this claim is made
after they make it.
Isn't C++ the underlying language that both operating systems are written
in?
And if so, wouldn't both operating systems be *equal* in their
susceptibility to buffer overrun attacks, for example, due to the pitfalls
of C-string functions like 'strcpy', 'strcat' in C++?
Perhaps this is narrowing things down to such a simplistic and specific
example, but I feel it is a fair question to ask. Thank you for sharing any
comments on this.
Best regards,
Ken Pergola
--
http://www.piclist.com hint: To leave the PICList

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[Ken Pergola]

Post by Shawn Wilton
For the most part, security is a matter of the programmer writing the
code. If the programmer knows what they're doing, then all is well. If
they don't know how to write secure code, then you get in to
problem areas.

Hi Shawn,

Thanks for your comments. Do you think people who make this claim are
talking about the OS only, the applications that run on top of the OS, or
both? It's difficult to wade through all these claims. Does the Linux camp
know how to write more secure code than the Microsoft camp? Or is it that
programmers were less hacker-aware when these operating systems were
written?

Best regards,

Ken Pergola

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[David Koski]

On Fri, 30 Apr 2004 20:10:51 -0400

Post by Ken Pergola

Post by Shawn Wilton
For the most part, security is a matter of the programmer writing the
code. If the programmer knows what they're doing, then all is well. If
they don't know how to write secure code, then you get in to
problem areas.

Hi Shawn,
Thanks for your comments. Do you think people who make this claim are
talking about the OS only, the applications that run on top of the OS, or
both?

I don't know what the "people who make this claim are talking about" but it is
in fact both. Consider for example the typical virus or worm. It easily infects
Windows computers partly because the email cients (applications) make it easy to
execute code (programs) that are received by email. And when a virus or worm is
executed, a lax security model (the OS) often allows it to take over.

Post by Ken Pergola
It's difficult to wade through all these claims. Does the Linux camp
know how to write more secure code than the Microsoft camp?

Microsoft is driven by their bottom line and people tend to buy what they are
used to, has a pretty look, is cool, etc. So why should M$ spend the time
(money) to improve Windows security unless the consumer demands it? And I think
that may be happening now.

Post by Ken Pergola
Or is it that programmers were less hacker-aware when these operating systems
were written?

I'm not sure what you mean but FYI, Linux is modeled after Unix which has been
arround a lot longer than Windows.

David

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[Ken Pergola]

Post by David Koski
I'm not sure what you mean but FYI, Linux is modeled after Unix
which has been around a lot longer than Windows.

Hi David,

Thanks for your comments. Yes, definitely, I'm aware of this fact. What I
meant to convey was a mindset: How can one write defensive code or secure
code if one does not know what they are defending against? Besides the money
factor, there's the possibility that people just didn't anticipate some of
these security problems. It appears to be an evolutionary process, and there
has been a lot of emphasis on improvement in security recently for obvious
reasons.

Thanks again,

Ken Pergola

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[James Caska]

I think it has more to do with the fact that people are attacking
windows not linux just 'because'

I would have thought having access to the source-code would be the
perfect thing to use to find security flaws that can be exploited. It's
like trying to crack an encryption algorithm while holding the key and
the source in your hand.

At least hiding the flaws makes them harder to find.. But then windows
seems to have a lot of flaws to find!

JC


-----Original Message-----
From: pic microcontroller discussion list
[mailto:PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org] On Behalf Of Ken Pergola
Sent: Friday, April 30, 2004 7:02 PM
To: PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org
Subject: Re: [OT:] Windows/Linux security

Post by David Koski
I'm not sure what you mean but FYI, Linux is modeled after Unix which
has been around a lot longer than Windows.

Hi David,

Thanks for your comments. Yes, definitely, I'm aware of this fact. What
I meant to convey was a mindset: How can one write defensive code or
secure code if one does not know what they are defending against?
Besides the money factor, there's the possibility that people just
didn't anticipate some of these security problems. It appears to be an
evolutionary process, and there has been a lot of emphasis on
improvement in security recently for obvious reasons.

Thanks again,

Ken Pergola

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[Ken Pergola]

Thanks for the comments James -- definitely a polarized topic.

Best regards,

Ken Pergola

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[Jake Anderson]

<snip>

Post by David Koski
Windows computers partly because the email cients (applications) make it easy to
execute code (programs) that are received by email. And when a virus or worm is
executed, a lax security model (the OS) often allows it to take over.

uhh no
microsoft has made it basically impossible to open any form of attachment in
outlook and outlook express.
prior to this you still had to manually execute the program you recieved.
there was a bug or 2 that allowed malformed mails to cause attached programs
to run but this has been fixed for years.

all current worms for windows clients opperate in user space. all they do is
read the current users mailbox and send mails to all they find, some also
scan the hard drive for mail addressess too.
There is no possible mechanism of stopping software from doing that as it is
perfictally valid operations that some user may want.

the reason windows propigates so many worms and virii is a) its the largest
single target. b) the users.
your average linux user will not randomly execute programs.
if your average windows user gets something in their inbox saying hahaha.exe
they will click on it.
microsoft has been forced to basically remove functionality from their
programs to protect the users from their own naievity
I write software, I currently have to send that software out to our offices
in 3 different formats due to the combinations of virus scanners and outlook
restrictions they have.

dont blame microsoft for the idjit users

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[David Koski]

On Sat, 1 May 2004 12:09:32 +1000

Post by Jan-Erik Soderholm

Post by David Koski
Windows computers partly because the email cients (applications) make it

easy to

Post by David Koski
execute code (programs) that are received by email. And when a virus or

worm is

Post by David Koski
executed, a lax security model (the OS) often allows it to take over.

uhh no
microsoft has made it basically impossible to open any form of attachment in

I'm glad to hear M$ has made improvements in Outlook. To be fair, I hardly use
Windows any more. I would like to hear from others how difficult it is to
execute code received from Internet sources, be it email or not, especially
compared to a Linux equivalent

FYI, to execute code received in an email using Sylpheed, my email clinent for
Linux, I have to deliberately select the attachments tab, right click on the
attachemnt, select "save as", then go to the folder where it was saved and then
set the execute bit before I can execute it. I would think the average "idjit"
would have trouble getting that far. Actually, I have never done it so I have
not tested it.

<snip>

David

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[Jake Anderson]

in outlook you need to edit the registry in order to even have the ability
to save a program out
outlook express you need to find a buried security tab disable it before you
are able to save it out
once you have disabled the security you can run the program from outlook/e
with a few clicks but if sombody is going to run it they will run it wether
they have to save it or not.

microsoft has had patches out for all the major worms that have gone around
recently several months before the exploit.

the problem is the users dont update
windows can be made secure to the point you basically cant use it if your
that way inclined.
windows is roughly as secure as linux when they are both run by compitent
people up to a level good enough for most of the population.

personally i trade a little "security" for the ability to do just about
anything i want with a nice menu, good help system, and widespread hardware
support.

-----Original Message-----
From: pic microcontroller discussion list
[mailto:PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org]On Behalf Of David Koski
Sent: Saturday, May 01, 2004 1:05 PM
To: PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org
Subject: Re: [OT:] Windows/Linux security


On Sat, 1 May 2004 12:09:32 +1000

Post by Jan-Erik Soderholm

Post by David Koski
Windows computers partly because the email cients (applications) make it

easy to

Post by David Koski
execute code (programs) that are received by email. And when a virus or

worm is

Post by David Koski
executed, a lax security model (the OS) often allows it to take over.

uhh no
microsoft has made it basically impossible to open any form of attachment in

I'm glad to hear M$ has made improvements in Outlook. To be fair, I hardly
use
Windows any more. I would like to hear from others how difficult it is to
execute code received from Internet sources, be it email or not, especially
compared to a Linux equivalent

FYI, to execute code received in an email using Sylpheed, my email clinent
for
Linux, I have to deliberately select the attachments tab, right click on the
attachemnt, select "save as", then go to the folder where it was saved and
then
set the execute bit before I can execute it. I would think the average
"idjit"
would have trouble getting that far. Actually, I have never done it so I
have
not tested it.

<snip>

David

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[Robert B.]

If windows is roughly equivalent in security to linux, then why are there so
few viruses that break down *nix servers? Is this merely a function of the
virus-writers targeting the unsusupecting windows non-technical users? Or
is it because windows does in fact have more exploitable security holes. I
tend to think it's the latter, because if I were a virus writer and wanted
to do serious damage it would make sense to target the servers which hold
the internet together (i.e. unix variety) as opposed to the end users
windows machines. Speaking from personal experience, my windows machines
regularly get infected with nuisance viri and the occasional more serious
virus, but I have yet to experience any such nuisance in my freebsd machine.

As far as ouhouse express works, I have no problem running attachments from
the attachment menu.


----- Original Message -----
From: "Jake Anderson" <grooveee-***@public.gmane.org>
To: <PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org>
Sent: Friday, April 30, 2004 10:48 PM
Subject: Re: [OT:] Windows/Linux security

Post by Jake Anderson
in outlook you need to edit the registry in order to even have the ability
to save a program out
outlook express you need to find a buried security tab disable it before you
are able to save it out
once you have disabled the security you can run the program from outlook/e
with a few clicks but if sombody is going to run it they will run it wether
they have to save it or not.
microsoft has had patches out for all the major worms that have gone around
recently several months before the exploit.
the problem is the users dont update
windows can be made secure to the point you basically cant use it if your
that way inclined.
windows is roughly as secure as linux when they are both run by compitent
people up to a level good enough for most of the population.
personally i trade a little "security" for the ability to do just about
anything i want with a nice menu, good help system, and widespread hardware
support.
-----Original Message-----
From: pic microcontroller discussion list
Sent: Saturday, May 01, 2004 1:05 PM
Subject: Re: [OT:] Windows/Linux security
On Sat, 1 May 2004 12:09:32 +1000

Post by Jan-Erik Soderholm

Post by David Koski
Windows computers partly because the email cients (applications) make it

easy to

Post by David Koski
execute code (programs) that are received by email. And when a virus or

worm is

Post by David Koski
executed, a lax security model (the OS) often allows it to take over.

uhh no
microsoft has made it basically impossible to open any form of

attachment

Post by Jake Anderson
in
I'm glad to hear M$ has made improvements in Outlook. To be fair, I hardly
use
Windows any more. I would like to hear from others how difficult it is to
execute code received from Internet sources, be it email or not, especially
compared to a Linux equivalent
FYI, to execute code received in an email using Sylpheed, my email clinent
for
Linux, I have to deliberately select the attachments tab, right click on the
attachemnt, select "save as", then go to the folder where it was saved and
then
set the execute bit before I can execute it. I would think the average
"idjit"
would have trouble getting that far. Actually, I have never done it so I
have
not tested it.
<snip>
David
--
http://www.piclist.com hint: To leave the PICList
--
http://www.piclist.com hint: To leave the PICList

--
http://www.piclist.com hint: PICList Posts must start with ONE topic:
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

[David Koski]

On Fri, 30 Apr 2004 23:08:03 -0500

Post by Robert B.

Post by Jake Anderson
in outlook you need to edit the registry in order to even have the ability
to save a program out
outlook express you need to find a buried security tab disable it before
you are able to save it out
once you have disabled the security you can run the program from outlook/e
with a few clicks but if sombody is going to run it they will run it
wether they have to save it or not.

<snip>

Post by Robert B.
Speaking from personal experience, my windows machines
regularly get infected with nuisance viri and the occasional more serious
virus, but I have yet to experience any such nuisance in my freebsd machine.

Okay, I'm confused. Surely you are not jumping through the hoops indicated above
just to get "viri". How do you (and every other Windows user connected to
Internet) get them?

<snip>

David

--
http://www.piclist.com hint: PICList Posts must start with ONE topic:
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

[Jake Anderson]

by not patching

i run windows 2k and 2k server on all the machines here.
i virus scan every once in a while.
i have never been infected with a virus.
i get sent on average 15 virus emails a day, same for dad.

If you have patched outlook express or outlook up to date you cannot run
programs without disabling the security.
if you can run programs or hell even save .pif .exe whatever files out of
the program you have either not patched or have disabled the "security".

About the worst that has happened is dads computer was infected with the
scumware "cool web search" looks like i forgot to patch his computer once
and it got in through a JVM vunerability.

by the way before anybody makes any snide remarks about patching linux is
just as bad, it wanted to install 400mb of patches for my mandrake install.

-----Original Message-----
From: pic microcontroller discussion list
[mailto:PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org]On Behalf Of David Koski
Sent: Saturday, May 01, 2004 2:49 PM
To: PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org
Subject: Re: [OT:] Windows/Linux security


On Fri, 30 Apr 2004 23:08:03 -0500

Post by Robert B.

Post by Jake Anderson
in outlook you need to edit the registry in order to even have the ability
to save a program out
outlook express you need to find a buried security tab disable it before
you are able to save it out
once you have disabled the security you can run the program from outlook/e
with a few clicks but if sombody is going to run it they will run it
wether they have to save it or not.

<snip>

Post by Robert B.
Speaking from personal experience, my windows machines
regularly get infected with nuisance viri and the occasional more serious
virus, but I have yet to experience any such nuisance in my freebsd machine.

Okay, I'm confused. Surely you are not jumping through the hoops indicated
above
just to get "viri". How do you (and every other Windows user connected to
Internet) get them?

<snip>

David

--
http://www.piclist.com hint: PICList Posts must start with ONE topic:
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

--
http://www.piclist.com hint: PICList Posts must start with ONE topic:
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

[Russell McMahon]

if you can ... save .pif .exe whatever files out of
the program you have either not patched or have disabled the "security".

That would be the "brain dead security" wouldn't it ? :-)

I think it's fine to be able to disable pif/exe/com/... running AND to be
able to stop them being even saved BUT when you can have only both or
neither it's ludicrous. Of course I don't want to have my email program
running exe files in the current environment - too much chance for a
mistake. But I certainly want to be able to save exe files I am sent. I
can't imagine why it HAS to be both or neither. Maybe that's been changed
since I last looked at the options.



RM

--
http://www.piclist.com hint: PICList Posts must start with ONE topic:
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

[Jake Anderson]

you can set the behaviour if you want to
personally i like being able to run the odd program from there
however when you click it it pops up asking you what action you wish to take
and warning that files can have viruses etc if you uncheck "ask me every
time" and click save then from then on it will prompt to save whenever you
click on a file of that type. you must disable the "child lock" security
first.
thats what it is really a child lock.


-----Original Message-----
From: pic microcontroller discussion list
[mailto:PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org]On Behalf Of Russell McMahon
Sent: Saturday, May 01, 2004 5:02 PM
To: PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org
Subject: Re: [OT:] Windows/Linux security

if you can ... save .pif .exe whatever files out of
the program you have either not patched or have disabled the "security".

That would be the "brain dead security" wouldn't it ? :-)

I think it's fine to be able to disable pif/exe/com/... running AND to be
able to stop them being even saved BUT when you can have only both or
neither it's ludicrous. Of course I don't want to have my email program
running exe files in the current environment - too much chance for a
mistake. But I certainly want to be able to save exe files I am sent. I
can't imagine why it HAS to be both or neither. Maybe that's been changed
since I last looked at the options.



RM

--
http://www.piclist.com hint: PICList Posts must start with ONE topic:
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

--
http://www.piclist.com hint: PICList Posts must start with ONE topic:
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

[John J. McDonough]

----- Original Message -----
From: "Robert B." <piclist-***@public.gmane.org>
Subject: Re: [OT:] Windows/Linux security

Post by Robert B.
If windows is roughly equivalent in security to linux, then why are there so
few viruses that break down *nix servers?

Server attacks are relatively rare anyway. There are a lot more desktops
out there to atttack, so it's a much more appealing target. And virtually
all of the desktops are Windows.

However, one place where there is a significant difference is in web
servers. Until about a year ago, the majority of the web servers were
Apache, with most of the rest being IIS. (IIS has changed it's name a
couple times in the past few years, but a rose by any other name....) About
a year or two ago, IIS passed Apache in numbers, but they are still roughly
equivalent.

However, for quite a while almost every day brought a new IIS vulnerability,
while Apache went literally YEARS between exploits. Recently the Apache
exploits have picked up some, and the IIS exploits have slowed, but this is
one area where the difference is still dramatic.

--McD

--
http://www.piclist.com hint: PICList Posts must start with ONE topic:
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

[John J. McDonough]

----- Original Message -----
From: "Jake Anderson" <grooveee-***@public.gmane.org>
Subject: Re: [OT:] Windows/Linux security

Post by Jake Anderson
uhh no
microsoft has made it basically impossible to open any form of attachment in
outlook and outlook express.

uhhhh ... no.

First of all, it has been possible to keep reasonably secure with Outlook
Express for a very long time now. However, the default settings used to be
just plain bad, and even though they have improved, they are still not what
they need to be. Interestingly, as best I can tell, Outlook still has some
problems that I don't know that you can work around.

As far as I know, there are three ways someone can nail you with an email.

The first is a buffer overflow vulnerability. Back when the only computers
on the net were Unix machines, this was the favorite. The famous "Internet
Worm" that got so much press a few years back worked this way. This isn't
so popular with Windows. I don't think it's necessarily harder to do in
Windows, it's just that Windows provides so many easier approaches.

The second is getting a brain-dead user to open an executable attachment.
Recent versions of OE have made this harder, but not impossible. What is
really annoying is that M$ has provided a convenient way for a hacker to
make an exe file look like a jpeg. This "feature" can be turned off, but it
takes mucking around in the registry to do it. A big difference here is
that many Windows users haven't found the clue bucket. Linux is such a pain
to install and configure that the totally brain dead user isn't going to be
running Linux in the first place, so it's a safe bet that almost all Linux
users will have enough sense not to do this.

The third avenue is through HTML email. HTML provides a rich set of tools
for exploiting the target system, although all of them are fairly hard to
use except for ActiveX controls. Some Linux clients will open email in HTML
and so are susceptible to exploits via JavaScript or Java (don't give me
that crap about the sandbox being secure - it ain't). But these exploits
are more difficult than ActiveX. Controlling HTML email and ActiveX are
quite possible in Outlook Express (less so in Outlook), but the settings are
scattered all over, so the average user isn't likely to get them right.
There have also been bugs in the HTML engines from time to time that are
exploitable without active content, although these have been fairly
infrequent.

Other than ActiveX, all of these are available on both Linux and Windows,
although some mail clients will not open HTML mail. These clients are more
popular on Linux than on Windows. The HTML is an especially nasty one
since, on clients with a preview pane, this sort of exploit can be activated
without actually opening the email.

Windows is a more popular attack target simply because there are so many
more Windows machines out there. Over 95% of the machines are Windows, so
Linux/Unix/Mac/VMS are much less appealing targets.

--McD

--
http://www.piclist.com hint: PICList Posts must start with ONE topic:
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

[Jake Anderson]

basically what it comes down to is mistakes.
both the linux and microsoft camp have good programmers (microsoft probbly
better as they actually have to get paid for it and sacked if they arent
good enough) but since its people putting it in mistakes will be made,
oversights, lazyness and all maner of other things. Yhe linux community
however has all their source code publicly availablle, so that if sombody
has made a mistake some other interested person who may just be browsing
through it would go "hey now thats odd" fix it and send it in to be
integrated. linux is also as a rule much quicker at resolving exploits and
problems that have been found, 24 hours is a long time for a security flaw
to be found and not yet resolved. Microsoft have been up around the months
level for some instances in the past, though they are much better now.

generally the opperating system is what people refer to when talking of
security, user level applications have only certain rights which the OS
gives to them. User level apps shouldnt be able to crash the OS or alter
anything in the system that could prevent it opperating normally. linux is
very good at this.

personally I use windows. its just easier.

-----Original Message-----
From: pic microcontroller discussion list
[mailto:PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org]On Behalf Of Ken Pergola
Sent: Saturday, May 01, 2004 9:59 AM
To: PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org
Subject: [OT:] Windows/Linux security


I was hoping someone could educate me on something that I see and hear a lot
of comments about. I hear a lot of claims that Linux is more secure than
Windows, but I don't usually see people elaborate on why this claim is made
after they make it.

Isn't C++ the underlying language that both operating systems are written
in?

And if so, wouldn't both operating systems be *equal* in their
susceptibility to buffer overrun attacks, for example, due to the pitfalls
of C-string functions like 'strcpy', 'strcat' in C++?

Perhaps this is narrowing things down to such a simplistic and specific
example, but I feel it is a fair question to ask. Thank you for sharing any
comments on this.

Best regards,

Ken Pergola

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[John J. McDonough]

----- Original Message -----
From: "Jake Anderson" <grooveee-***@public.gmane.org>
Subject: Re: ] Windows/Linux security

Post by Jake Anderson
integrated. linux is also as a rule much quicker at resolving exploits and
problems that have been found, 24 hours is a long time for a security flaw

The data would indicate otherwise. I think there is a "feeling" that the
open source community responds more quickly, but a study recently found that
when you look at the actual data, Linux exploits get resolved MUCH more
slowly than Windows.

The Linux crowd claimed this wasn't "fair" in that they fix "serious" bugs
more quickly than M$, although some of the less critical ones do lay around
for a while. And I suspect there is probably some truth to this. But it's
pretty subjective to try to categorize security problems as important or
unimportant.

One thing that I don't think has been mentioned, though. Windows code is
astonishingly complex. I think it is a lot harder to write secure code in a
Windows environment than in Linux. This is pretty subjective, though. The
other problem M$ has is a legacy of code that was written at a time when the
giant of Redmond didn't care about security.

In fact, I wonder why M$ cares even now. Ever since Windows 3.1,
Microsoft's customers have sent Redmond a strong message that pretty colors
and bouncing paper clips are a lot more important than stability and
security. And Microsoft listens to their customers. That's why they are so
successful. We geeks can sit around and whine about how awful that is, but
every day customers send millions of dollars to Microsoft to vote for the
status quo.

72/73 de WB8RCR http://www.qsl.net/wb8rcr
didileydadidah QRP-L #1446 Code Warriors #35

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[William Chops Westfield]

I hear a lot of claims that Linux is more secure than Windows, but I
don't usually see people elaborate on why this claim is made after
they make it.

I think the main thing that makes unix more secure is the underlying
multi-user privilege and file system structures. Windows is no less
secure than unix, if you always log into your unix system as root (the
maximum-priv user.) MOST unix applications run under the privs of a
single "normal" user and can therefore not infect or delete system
files, access privileged resources, etc. Excepting bugs. And
excepting that there are an awful lot of things that DO run as root,
installed and "maintained" by people who don't really understand them.

Newer windows operating systems are headed in this direction, of course.

BillW

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[Ken Pergola]

Thanks for taking the time to post your comments Jake Anderson, William
Westfield, and John McDonough. I appreciate everyone's insight.

It's definitely an intriguing topic. My angle of security was from an
outside hacker perspective.

Best regards and thanks again,

Ken Pergola

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[Carey Fisher - NCS]

<Resend with correct [OT:] tag; sorry, MS Outlook don't ya know...>

A major aspect of the "better security" claim is that Linux is "open source"
meaning the source code is
available for anyone to look at. The theory is that since it's open source,
security flaws will be found by the many people looking at the code.
Proprietary operating systems are not open to such wide scrutiny. There is
a good front page article in EETIMES (Mon, Apr 19,2004) that pretty much
discredits the idea that "...Linux's development process, which involves
thousands of individuals, makes it almost impossible for 'adversarial code'
to sneak through." (EETIMES).

Carey Fisher, K8VZ
Chief Technical Officer
New Communications Solutions, LLC
website: www.ncsradio.com

-----Original Message-----
From: pic microcontroller discussion list
Sent: Friday, April 30, 2004 7:59 PM
Subject: [OT:] Windows/Linux security
I was hoping someone could educate me on something that I see
and hear a lot
of comments about. I hear a lot of claims that Linux is more
secure than
Windows, but I don't usually see people elaborate on why this
claim is made

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[Carey Fisher - NCS]

A major aspect of the "better security" claim is that Linux is "open source"
meaning the source code is
available for anyone to look at. The theory is that since it's open source,
security flaws will be found by the many people looking at the code.
Proprietary operating systems are not open to such wide scrutiny. There is
a good front page article in EETIMES (Mon, Apr 19,2004) that pretty much
discredits the idea that "...Linux's development process, which involves
thousands of individuals, makes it almost impossible for 'adversarial code'
to sneak through." (EETIMES).

Carey Fisher, K8VZ
Chief Technical Officer
New Communications Solutions, LLC
website: www.ncsradio.com

-----Original Message-----
From: pic microcontroller discussion list
Sent: Friday, April 30, 2004 7:59 PM
Subject: [OT:] Windows/Linux security
I was hoping someone could educate me on something that I see
and hear a lot
of comments about. I hear a lot of claims that Linux is more
secure than
Windows, but I don't usually see people elaborate on why this
claim is made
after they make it.

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[Ken Pergola]

Hi Carey,

Thanks for your comments and for the link -- I'll check that out.

Best regards,

Ken Pergola

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[William Chops Westfield]

Post by Carey Fisher - NCS
A major aspect of the "better security" claim is that Linux is "open
source" meaning the source code is available for anyone to look at.

Unfortunately, this is probably largely bullshit. While anyone can
look at it, the number of people looking at it that are qualified to
evaluate complex security issues is probably similar to the number of
such people looking (internally) at microsoft code. And the crackers
get to look at it too, and they have more free time than secure
programming experts.

Meanwhile, the fact that joe user can look at the source code probably
does him no good at all. The world is luck if he's got enough brains
to watch for announcements of security fixes, and install them. God
knows a larger percentage of the "hits" on windows are on bugs that
have already been fixed...

I do feel SLIGHTLY warm and fuzzy that given a unix flaw (security or
otherwise), I DO have the option of modifying the source code myself to
provide a fix, rather than having to wait for a vendor to run it
through their qualification and release program. That "patch" can be
provided by myself, the community, or vendors, asynchronously to
"release process." (Go ahead; ask me how much I've learned about how
NOT to run release processed.)

BillW

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[M. Adam Davis]

There are many different factors, but the primary few issues are:

They each use fundamentally different security models. Windows is
geared towards both easy administration and many ease of use features.
The complication arises that something that is 'easy' to administer is
easy to misadminister since it doesn't require any special knowledge.
By default a regular user is an administrator who has direct access to
the computer. The complicating factor of many ease of use features
means that something that is easy for the user to do is easy for a
script to do. Since the script has full computer access, and many
programs run scripts without bounds then it's an easy target. I would
say the Linux has difficult administration (this is changing) and few
ease of use features (this is changing). As these two change you'll
find it targetted more frequently. However, the average linux user is
instructed to use root access very infrequently. Hopefully this will
not change (though it used to be that lindows ran in root, and so maybe
this will eventually happen).

They were built from different perspectives aiming at different users.
Windows is aiming for a person who doesn't share their computer with
other users simultaneously - in fact their business model depends on one
person - one license. Windows XP has all the features needed for
terminal service, minus some small amount of code and, of course, such
functionality is blocked. Linux (and unix in general) was designed from
the ground up as a general shared-use computer platform. As such the
security model was designed from the ground up for multiple, non-root
users. Furthermore applications are always run in very well defined
user spaces that prevent them from gaining any more priviledges than the
user has. This model even leads to being logged in as a user, and
running some programs (say email) as a lesser-capable user, and some
programs as a more capable user (administration). This, and other such
security features, is possible in windows, but it's not simple or
obvious to do because Microsoft doesn't want to make it hard for users.

There are other issues, but the two systems are slowly converging to the
same point since many people want linux to work in the space windows is
so good at, and Microsoft has forever been really trying to crack into
the space that linux works in so well. As they do so, they will also
learn from their mistakes and I suspect that windows won't become any
more susceptable, and Linux will become slightly more susceptable. One
of the nice things about linux now is that it's release cycle allows
patches to be used quickly, but only for those users who haven't become
complacent. Windows patches don't seem to come as quickly, and if they
aren't willing to fix something no one else is going to be able to
without the code.

-Adam

Post by Ken Pergola
I was hoping someone could educate me on something that I see and hear a lot
of comments about. I hear a lot of claims that Linux is more secure than
Windows, but I don't usually see people elaborate on why this claim is made
after they make it.
Isn't C++ the underlying language that both operating systems are written
in?
And if so, wouldn't both operating systems be *equal* in their
susceptibility to buffer overrun attacks, for example, due to the pitfalls
of C-string functions like 'strcpy', 'strcat' in C++?
Perhaps this is narrowing things down to such a simplistic and specific
example, but I feel it is a fair question to ask. Thank you for sharing any
comments on this.
Best regards,
Ken Pergola
--
http://www.piclist.com hint: To leave the PICList

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[Ken Pergola]

Thank you very much Adam for time and for your comments as well.

Best regards,

Ken Pergola

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[Matt Pobursky]

Post by Jake Anderson
what is it with america and not shipping outside USA? seems in every other
country if you arent willing to ship international your a nutter.

I can only speak from my own experience, but...

Almost every U.S. shipper makes it extremely difficult or expensive to
ship worldwide. The one notable exception this is the USPS -- but it
has one caveat -- you can't get reliable tracking or delivery
guarantees from them. About all you can get is a "hey -- we delivered
it" confirmation from them. If something goes awry in shipping, you are
pretty much on your own. Anyone who has had a parcel lost in the USPS
system can attest to this. They have no clue where anything is at a
given point in time, unlike Fedex or UPS.

I personally like Fedex the best for international shipments.
Everything is tracked along the way and they deal with customs/duties
in a reasonable manner, but they are not cheap. UPS's problems with
customs fees has been chronicled on the PIClist recently. USPS is a
pain for me, I have to go approximately 3 miles to the local Post
Office and wait in a long line while the minions behind the counter
move at the speed of mating elephants to ship anything with them.
Unlike a lot of countries, the mail carriers here do nothing but
deliver mail and parcels -- no pickup of parcels or even letters!

One other problem I've heard horror tales about is problems with
international fraud and banking problems, especially with credit cards
and the legal problems that ensue on a transaction gone bad. On a
personal shipment basis and with small orders, Paypal seems to be a
reasonable compromise. I can see how larger companies like Amazon.com
may opt to avoid international shipments where the percentage of fraud
or non-payments could kill their already razor-slim margins. Then add
in the complexities of legal issues across borders and legal systems
and you have a potentially large problem.

I certainly wish it were easier to ship internationally from the U.S.

Matt Pobursky
Maximum Performance Systems

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[M. Adam Davis]

I suspect that since we can get pretty much everything we would ever
want from someone inside the US that we don't even consider selling
outside the US. In fact there's likely a good percentage of americans
that have never sent or received anything outside the country. So to
many people it seems like there's natural barriers involved:

What's all this about customs? Tarrifs? Taxes?

Shippers only work with moving it from point A to B. The first time a
seller gets a call from overseas complaining about having to pay huge
customs fees because they filled out the declaration incorrectly may be
the last time they ship something overseas.

I don't want to have to deal with that. I get payment, I give it to the
shipper, and any further costs are absorbed by the customer. But most
americans are used to dealing with paying nothing more than the company
charged you for the product and shipping. Pretty soon states are going
to crack down even more on making people pay taxes on out of state
purchases, so we'll get used to it over time.

Folks elsewhere in the world are already familiar with all the
intricacies of their particular gov't, so you may think of all this as
second nature.

On topic, though, I did 'accidently' put 5 in my shopping basket and
'accidently' checked out. Fortunately I had some amazon gift
certificates, so I'm only paying $35 for them, but my wife will still
ask me what I got when it comes in. Perhaps I'll have to part with a
few - if so I'll let everyone on the list know.

-Adam

Post by Jake Anderson
anybody in the USA interested in picking some of these up and shipping them
to the rest of us scum? (ie people outside america) ;->
Promise we wont use them to simulate nuclear blasts or nothin.
what is it with america and not shipping outside USA? seems in every other
country if you arent willing to ship international your a nutter.
-----Original Message-----
From: pic microcontroller discussion list
Sent: Saturday, May 01, 2004 2:02 AM
Subject: Re: [OT:] Cybiko serial cable
Thanks for the info. Makes the ten buck deal even sweeter! I'll be
able to nail one to the wall and save some bench space. It should be
easier than firing up the computer just to check a datastream for
activity.
Bill
-----Original Message-----
From: pic microcontroller discussion list
Sent: Friday, April 30, 2004 11:55 AM
Subject: Re: [OT:] Cybiko serial cable

Post by William Bross
Can anybody who just purchased any classics tell us if the serial cable
comes with the unit and if not, what type connector does the unit need?
Bill

The classic model does come with a serial cable. It has some sort of
proprietary looking connector on the Cybiko end and a 9 pin d-sub on the
other end. Tom
--
http://www.piclist.com hint: To leave the PICList
--
http://www.piclist.com hint: To leave the PICList
--
http://www.piclist.com hint: To leave the PICList

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[Howard Winter]

Post by M. Adam Davis
I suspect that since we can get pretty much everything

we would ever want from someone inside the US

...except of course decent beer, decent chocolate,
Branston Pickle, and Marmite!

:-)))

Howard Winter
St.Albans, England

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[John Pearson]

Okay, I am getting confused. Someone said the Classic is available for 14.95
from toyrus.com, and the extreme for 9.99 from Amazon. I ordered a couple
from Amazon, but I don't really know what I ordered, and I can't find any
Classics for 14.95.

Anyway, someone also mentioned shipping out of US. Well, I will ship out
some Cybikos if someone will ship me a couple Cassio PV-S1600 PDAs

John

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[Richard Kendrick]

Yes, it does come with a serial cable. It is a 6 foot long cable with a
DB9 on one end and a small proprietary connector that plugs into the
unit.

-----Original Message-----
From: pic microcontroller discussion list
[mailto:PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org] On Behalf Of William Bross
Sent: Friday, April 30, 2004 8:21 AM
To: PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org
Subject: Re: [OT:] Cybiko serial cable

Can anybody who just purchased any classics tell us if the serial cable
comes with the unit and if not, what type connector does the unit need?

Bill

-----Original Message-----
From: pic microcontroller discussion list
[mailto:PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org] On Behalf Of John Ferrell
Sent: Thursday, April 29, 2004 7:37 PM
To: PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org
Subject: Re: [OT:] Cybiko serial cable


ToysRus.com, direct or through Amazon.com

John Ferrell
http://DixieNC.US

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

--
http://www.piclist.com hint: To leave the PICList
mailto:piclist-unsubscribe-request-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org

[Michael Olson]

Amazon.com

-- Mike Olson

Post by Shawn Wilton
Where are they on closeout?

Post by John Ferrell
They are $9.95 now...
I got my order. The two I opened seem pretty neat. 900mhz
communications,
tiny keyboard, LCD.
I am not sure all of the software is still available for the Classics
though, I am still looking. I may have to start begging!
It is definately worth the close out prices though.
John Ferrell
http://DixieNC.US
----- Original Message -----
Sent: Thursday, April 22, 2004 10:02 AM
Subject: Re: [OT:] Cybiko serial cable

Post by John Ferrell
Classics are currently $14.99US at Toysrus.com as a clearence item.
I ordered a few out of curiousity...
John Ferrell
http://DixieNC.US
----- Original Message -----
Sent: Wednesday, April 21, 2004 9:56 PM
Subject: Re: [OT:] Cybiko serial cable

Post by Sergio Masci
Also xtremes seem to be less reliable than the classics. I bought a

couple

Post by John Ferrell
of

Post by Sergio Masci
classics about a year ago to experiment with. My son regularly uses one

to

Post by John Ferrell
play

Post by Sergio Masci
MP3s on. Two of his friends decided to buy extremes. These have both had

to be

Post by Sergio Masci
replaced after developing faults.
Regards
Sergio Masci
--
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

--
http://www.piclist.com#nomail Going offline? Don't AutoReply us!

--
http://www.piclist.com#nomail Going offline? Don't AutoReply us!

--
http://www.piclist.com#nomail Going offline? Don't AutoReply us!
email listserv-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org with SET PICList DIGEST in the body

[Ken Pergola]

Post by John Ferrell
I am not sure all of the software is still available for the Classics
though, I am still looking. I may have to start begging!

Hi John,


Just mosey on down to:

http://www.cybiko.com/index.asp?go=cybiko

That's the Cybiko classic page. All the files are there.

If you have not done so already, don't forget to pick up the VTTerm.app file
(serial terminal emulator) that Jeff Frohwein created. You can use this to
display ASCII serial data from a PIC on your Cybiko screen. That's the
program I used in conjunction with the Master SPI Bus and Master I2C/SMBus
engines (PIC firmware).


To get VTTerm.app:

http://www.devrs.com/cybiko/

Search down the page for:

01/22 - VTTerm v1.2 - Posted by: Jeff Frohwein


Best regards,

Ken Pergola

P.S. The Cybiko classic is a pretty cool device from a technical standpoint.
As you probably have discovered, the Cybiko classic gives credence to the
adage: "Don't judge a book by its cover." There's some serious hardware in
the Cybiko!

--
http://www.piclist.com#nomail Going offline? Don't AutoReply us!
email listserv-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org with SET PICList DIGEST in the body

[John Ferrell]

Thanks Ken,
I had found the Classic link and I am just starting to find my way into the
Cybiko world. I was really only looking for a cheap pic io device, this
seems a lot more.

I am going to have to spend more time in toy departments...
The terminal program looks like something that will be very handy.

John Ferrell
http://DixieNC.US

----- Original Message -----
From: "Ken Pergola" <no_spam-c1EBYUxPPa5+cjeuK/***@public.gmane.org>
To: <PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org>
Sent: Thursday, April 29, 2004 7:52 PM
Subject: Re: [OT:] Cybiko serial cable

Post by Ken Pergola

Post by John Ferrell
I am not sure all of the software is still available for the Classics
though, I am still looking. I may have to start begging!

Hi John,
http://www.cybiko.com/index.asp?go=cybiko
That's the Cybiko classic page. All the files are there.
If you have not done so already, don't forget to pick up the VTTerm.app file
(serial terminal emulator) that Jeff Frohwein created. You can use this to
display ASCII serial data from a PIC on your Cybiko screen. That's the
program I used in conjunction with the Master SPI Bus and Master I2C/SMBus
engines (PIC firmware).
http://www.devrs.com/cybiko/
01/22 - VTTerm v1.2 - Posted by: Jeff Frohwein
Best regards,
Ken Pergola
P.S. The Cybiko classic is a pretty cool device from a technical standpoint.
As you probably have discovered, the Cybiko classic gives credence to the
adage: "Don't judge a book by its cover." There's some serious hardware in
the Cybiko!
--
http://www.piclist.com#nomail Going offline? Don't AutoReply us!

--
http://www.piclist.com#nomail Going offline? Don't AutoReply us!
email listserv-Xn+YjCi86RQ8qNwGfFDJ/***@public.gmane.org with SET PICList DIGEST in the body

[Dominic Stratten]

The Cybiko extreme uses a USB connection not an RS232 IIRC.

I bought a crate of the Classics over the Extremes (although I could have
got them for roughly the same price) as they have standard RS232 built in.

Buy a Classis - they are only around 15ukp/15usd if you shop around and well
worth the money :)

Regards

Dom

-----Original Message-----
From: pic microcontroller discussion list
[mailto:PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org]On Behalf Of Tom
Sent: 22 April 2004 00:14
To: PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org
Subject: [OT:] Cybiko serial cable


Ken Pergola (and maybe a few others) mentioned recently about using a
Cybiko Classic hand-held computer for a cheap and simple serial terminal.

Question for anyone who knows about Cybikos: what cable can be used with a
Cybiko Xtreme for serial data? Or is it possible?

Thanks!
Tom

--
http://www.piclist.com hint: PICList Posts must start with ONE topic:
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

[Jan-Erik Soderholm]

Post by Ben Hencke
I would really reccomend that you stay away from hashing at all. Never
hash a serial number that already fits into a reasonable ammount of
space. Hashing is really only useful when trying to take a
large key of data (ie a string, file, whole record, etc) and make it into a
new key that can be easily sorted/indexed/etc.

A hash index isn't ment to keep records "sorted" at all.
A hash index should be used when you need optimum performance
in retreiving a single record from a table. A properly designed
hash table/index should be able to retreive a specific fully
qualified record in one single I/O. A b-tree needs as many I/O's
as there are currently "levels" in the tree.

On the other hand, if you need range retreivals (using part of the key),
a B-tree is far better and a hash index sucks...

This applies to databases in general, maybe not to the case at hand.

So there is no "best" here, it, as usual, depends.
And the size of the key has not much to do with this decision.

Post by Ben Hencke
A card # that is 1 to 10 milion
already fits nicely into 24 bits. Hashing this small ammount of data
will only complicate things.

Maybe, but a hash it gives the fastest retreival of the record.

Jan-Erik

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

[Ben Hencke]

Post by Ben Hencke

Post by Ben Hencke
I would really reccomend that you stay away from hashing at all.

Never

Post by Ben Hencke
hash a serial number that already fits into a reasonable ammount of
space. Hashing is really only useful when trying to take a
large key of data (ie a string, file, whole record, etc) and make

it into a

Post by Ben Hencke
new key that can be easily sorted/indexed/etc.

A hash index isn't ment to keep records "sorted" at all.

A "hash" can be used for all sorts of things including as keys in a
sorted array when it is impractical to sort the original key itself.

A hash table is sorted with the exception of the collisions. You use a
hashing function to reduce the data into a smaller key or index and
store the data (or pointers to the data) in a simple table or array.
Each element is sorted by its index/hash by virtue of being in an
array. You can access any record in one i/o because with its hash value
you know where in the array it is.

See
http://ciips.ee.uwa.edu.au/~morris/Year2/PLDS210/hash_tables.html
It has a nice animation too :)

IIRC the max simultaneous number of records required was 100,000 or so.
A minimum hash table that could handle the required number of records
would need a 17bit hash value. Thats still something like a 70% overlap
with a perfect hash function.

Most hash tables are an array of pointers to records, but for
optimization sake, I would group the whole record into the array
instead.

The OP won't be able to use a flat or rehash collision handling method
for a hash table because every deletion would require a rehash of the
table. You can optimize the rehash a little with some flags, but it
will still be a nightmare wrapped in a bad dream. So linked lists or
some other method would have to be used for each record that had a
collision further adding to the layers of complexity and storage
requirements.

Post by Ben Hencke
A hash index should be used when you need optimum performance
in retreiving a single record from a table. A properly designed
hash table/index should be able to retreive a specific fully
qualified record in one single I/O. A b-tree needs as many I/O's
as there are currently "levels" in the tree.
On the other hand, if you need range retreivals (using part of the key),
a B-tree is far better and a hash index sucks...
This applies to databases in general, maybe not to the case at hand.
So there is no "best" here, it, as usual, depends.

Agreed :-)

Post by Ben Hencke
And the size of the key has not much to do with this decision.

I disagree, the size of the key has a lot to do with picking the
algorithm to use. If the key was 8 bits, no one would be arguing about
what method to use ;-) If the key was 10k of data (maybe a data file or
sound sample), you would want to hash that down to something
reasonable.

Post by Ben Hencke

Post by Ben Hencke
A card # that is 1 to 10 milion
already fits nicely into 24 bits. Hashing this small ammount of

data

Post by Ben Hencke
will only complicate things.

Maybe, but a hash it gives the fastest retreival of the record.

A flat array with an element for each of the 10 million cards would
give the fastest access as even hash tables need to deal with
collisions. A hash table that was sized to fit would have horrible
performance when it approaches capacity. You would need significant
excess table size to get decent performance while running at capacity.

I don't think that fast access times are required. The OP is using this
for a vending machine so taking a second to look it up is probably
acceptable.

It all really depends on the application. I favor the sorted linked
list because it is simple to implement and very storage efficient. In
this application it is also easy to speed up with a few tweaks. I
wouldn't use it for anything that required near real-time lookup.

ttyl,
Ben Hencke




__________________________________
Do you Yahoo!?
Yahoo! Photos: High-quality 4x6 digital prints for 25"
http://photos.yahoo.com/ph/print_splash

--
http://www.piclist.com hint: The PICList is archived three different
ways. See http://www.piclist.com/#archives for details.

[Kresho]

G'day list.

Thanks for everyone's ideas. The info on hashing and b-tree's has been
enlightening!

I have finally decided on a scheme which has taken on some of the points
raised in the thread but uses a practical approach which i think will work,
is scalable in some sort of way and should be easy to implement.

There are up to 10million different cards, but it is nigh impossible that
that many cards would be used within any realistic span of time (months).

Using a 128MB MMC:

I'm going to dedicate 40million bytes of memory to a list of 10million 32bit
links for each card. This will allow the fastest possible time to access a
link as it's just a flat, linear array.

That leaves 88MB (or thereabouts anyway) for data. This data space will be
divided into appropriately sized data chunks. For the purpose of testing
this idea, i'm assuming each chunk of data is 64 bytes: this can hold the
card number (4 bytes) and up to 60 bytes of other info like the time a token
is set, the time it is retrieved, etc. This also allows for multiple tokens
or other info if that need ever arises. 88MB / 64bytes = 1.375 million
entries at any given time. I can live with that.

When a card is presented, the link table is looked up. If it's empty, a new
link is created to the next location in the data space and that data space
is written with appropriate data. If it's already got a valid link, that
link is updated to the next location in the data space and the old data
(from the old link) + new additions / modifications are copied to the data
space. When data space is full it just rolls over and starts again.

So, this should work quite quickly, the worst case being reading an old
link, creating a new link, reading the old data and copying it to the new
data area. It should also provide even wear of the data space so that write
endurance becomes a non-issue. And it should need no other maintenance.

Does that sound like a plausible idea?

Rgds,

Kresho.

--
http://www.piclist.com hint: PICList Posts must start with ONE topic:
[PIC]:,[SX]:,[AVR]: ->uP ONLY! [EE]:,[OT]: ->Other [BUY]:,[AD]: ->Ads

[Scott Thomas]

I am having some trouble with data retention on an 18F2320. I am
writing 32 values to EEPROM starting at EEADR = 0. Immediately after
writing the values I can read them into an array and everything works
fine. However, if I perform a soft reset or a power off then back on I
read all zero's from EEPROM.

Have I missed something obvious?

--

Thanks,

Scott Thomas

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

[Andre Thomas]

Perhaps something to do with initializing ?

Write one byte and see if you can still read back the other 31 or so...
Do they still return zero's ?

-----Original Message-----
From: pic microcontroller discussion list
[mailto:PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org] On Behalf Of Scott Thomas
Sent: Tuesday, April 27, 2004 6:16 AM
To: PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org
Subject: [PIC:] 18F2320 EEPROM Problems


I am having some trouble with data retention on an 18F2320. I am
writing 32 values to EEPROM starting at EEADR = 0. Immediately after
writing the values I can read them into an array and everything works
fine. However, if I perform a soft reset or a power off then back on I
read all zero's from EEPROM.

Have I missed something obvious?

--

Thanks,

Scott Thomas

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics


************************************************************
Scanned by @lantic IS Virus Control Service
This message was scanned for viruses and dangerous content. @lantic
Internet Services (Pty) Ltd. - http://www.lantic.net eScan for
Windows-based PCs - http://www.escan.co.za

If you have received a message marked in the subject line
as [SPAM] please note that according to our MailScanner,
this message has all the attributes of Unsolicited
Commercial Email (UCE). If the message has however been
marked incorrectly, please send a query to abuse-rgPbjVWyLIysTnJN9+***@public.gmane.org
************************************************************

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

[James Caska]

Any chance you have accidently fallen through to some sort of ERASE_ALL
command during code startup?

James Caska
www.muvium.com
uVM - 'Java Bred for Embedded'


-----Original Message-----
From: pic microcontroller discussion list
[mailto:PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org] On Behalf Of Andre Thomas
Sent: Monday, April 26, 2004 11:41 PM
To: PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org
Subject: Re: [PIC:] 18F2320 EEPROM Problems

Perhaps something to do with initializing ?

Write one byte and see if you can still read back the other 31 or so...
Do they still return zero's ?

-----Original Message-----
From: pic microcontroller discussion list
[mailto:PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org] On Behalf Of Scott Thomas
Sent: Tuesday, April 27, 2004 6:16 AM
To: PICLIST-HvpawkHQRZp23lw5o+***@public.gmane.org
Subject: [PIC:] 18F2320 EEPROM Problems


I am having some trouble with data retention on an 18F2320. I am
writing 32 values to EEPROM starting at EEADR = 0. Immediately after
writing the values I can read them into an array and everything works
fine. However, if I perform a soft reset or a power off then back on I
read all zero's from EEPROM.

Have I missed something obvious?

--

Thanks,

Scott Thomas

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics


************************************************************
Scanned by @lantic IS Virus Control Service
This message was scanned for viruses and dangerous content. @lantic
Internet Services (Pty) Ltd. - http://www.lantic.net eScan for
Windows-based PCs - http://www.escan.co.za

If you have received a message marked in the subject line
as [SPAM] please note that according to our MailScanner,
this message has all the attributes of Unsolicited
Commercial Email (UCE). If the message has however been
marked incorrectly, please send a query to abuse-rgPbjVWyLIysTnJN9+***@public.gmane.org
************************************************************

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

[Ken Pergola]

Hi Scott,

See my post to Peter Anderson a few minutes ago. There is also an errata
document that details the same problem (EEPROM and LVD) on the
PIC18F2220/2320/4220/4320.

(PIC18F2220/2320/4220/4320 Rev. B2 Silicon/Data Sheet Errata - DSDS80156D)

There might also be later versions of errata for these devices on the
Microchip web site.

Might be worth trying the work-around just to see if that changes anything.

Best regards,

Ken Pergola

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

[Michael Rigby-Jones]

-----Original Message-----
Sent: 27 April 2004 02:46
Subject: Re: [PIC:] Sorting through millions of numbers
G'day list.
Thanks for everyone's ideas. The info on hashing and b-tree's
has been enlightening!
I have finally decided on a scheme which has taken on some of
the points raised in the thread but uses a practical approach
which i think will work, is scalable in some sort of way and
should be easy to implement.
There are up to 10million different cards, but it is nigh
impossible that that many cards would be used within any
realistic span of time (months).
I'm going to dedicate 40million bytes of memory to a list of
10million 32bit links for each card. This will allow the
fastest possible time to access a link as it's just a flat,
linear array.
That leaves 88MB (or thereabouts anyway) for data. This data
space will be divided into appropriately sized data chunks.
For the purpose of testing this idea, i'm assuming each chunk
of data is 64 bytes: this can hold the card number (4 bytes)
and up to 60 bytes of other info like the time a token is set,
the time it is retrieved, etc. This also allows for multiple
tokens or other info if that need ever arises. 88MB / 64bytes
= 1.375 million entries at any given time. I can live with that.
When a card is presented, the link table is looked up. If it's
empty, a new link is created to the next location in the data
space and that data space is written with appropriate data. If
it's already got a valid link, that link is updated to the
next location in the data space and the old data (from the old
link) + new additions / modifications are copied to the data
space. When data space is full it just rolls over and starts again.
So, this should work quite quickly, the worst case being
reading an old link, creating a new link, reading the old data
and copying it to the new data area. It should also provide
even wear of the data space so that write endurance becomes a
non-issue. And it should need no other maintenance.
Does that sound like a plausible idea?

How do you know where the next 64byte chunk of free space is located? If you
have an index pointer, it's going to get a lot of writes.

Regards

Mike




=======================================================================
This e-mail is intended for the person it is addressed to only. The
information contained in it may be confidential and/or protected by
law. If you are not the intended recipient of this message, you must
not make any use of this information, or copy or show it to any
person. Please contact us immediately to tell us that you have
received this e-mail, and return the original to us. Any use,
forwarding, printing or copying of this message is strictly prohibited.
No part of this message can be considered a request for goods or
services.
=======================================================================
Any questions about Bookham's E-Mail service should be directed to
postmaster-***@public.gmane.org

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

[Alan B. Pearce]

Post by Michael Rigby-Jones
How do you know where the next 64byte chunk of free space is located?
If you have an index pointer, it's going to get a lot of writes.

The trick here is to make use of a sneaky feature of MMC cards. they come
formatted as a FAT file system, or can be easily formatted as such by any
USB reader that handles them. You can then use the FAT block number as part
of the key pointer to find a particular field you want to reference. It does
involve putting FAT file handling code into the PIC, but that has been done
before.

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

[Jan-Erik Soderholm]

Post by Scott Thomas
I am having some trouble with data retention on an 18F2320. I am
writing 32 values to EEPROM starting at EEADR = 0. Immediately after
writing the values I can read them into an array and everything works
fine. However, if I perform a soft reset or a power off then
back on I read all zero's from EEPROM.

Now, since we have no actualy source code that shows what
you are talking about, I can only provide some "guess-ware"...

Post by Scott Thomas
Immediately after writing the values I can read them into an
array and everything works fine

Is that the same array used to store the original data ?
Are you sure you don't just "see" the old data ?

A short ASM code example thatshows the problem would
help a lot.

Post by Scott Thomas
Have I missed something obvious?

You have probably missed *something*, but if it was
obvious ? Who knows... :-) :-)

Jan-Erik.

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

[Scott Thomas]

Ok... Since no one responded with a "did you remember to stand on one
foot while touching your nose?" sort of answer, I'll assume there isn't
an undocumented trick I must perform. ;-)

I will check a couple things and get back, perhaps with some source even!

Thanks
Scott

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

[Peter Anderson]

Scott,

I had an odd problem with the PIC18F1320.

I was writing to EEPROM and then verifying. I
actually had to put it in a loop until it was
successful and there were times when it took over a
thousand writes.

I tried another style of programmer with much the same
result.

I went back to using the ICD2 which I had used to
debug the design and used it as a programmer and the
write was successful the first try everytime.

I am somewhat embarrassed to note this as it simply
doesn't make sense.

The design is non critical (an LCD and data collection
kit) and I decided to go ahead and field the design
and continue to use the ICD2 to program this design.

But, I would guess I burned up 35 hours trying to find
the problem. I saw another unexplained oddity with
the 1320 which left me a bit leery.

Peter H Anderson, http://www.phanderson.com

Post by Scott Thomas
Ok... Since no one responded with a "did you
remember to stand on one
foot while touching your nose?" sort of answer, I'll
assume there isn't
an undocumented trick I must perform. ;-)
I will check a couple things and get back, perhaps
with some source even!
Thanks
Scott
--
http://www.piclist.com hint: The list server can
filter out subtopics
(like ads or off topics) for you. See

http://www.piclist.com/#topics





__________________________________
Do you Yahoo!?
Win a $20,000 Career Makeover at Yahoo! HotJobs
http://hotjobs.sweepstakes.yahoo.com/careermakeover

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

[Ken Pergola]

...I had an odd problem with the PIC18F1320...

Hi Peter,

I remember your post about this. That was a strange problem. Did you happen
to see the errata document regarding the EEPROM and the LVD issue? See
below:


2004 Microchip Technology Inc. DS80175B-page 1

Module: Data EEPROM
-------------------
When writing to the data EEPROM, the contents of
the data EEPROM memory may not be written as
expected.

Work around
-----------
Either of two work arounds can be used:

1. Before beginning any writes to the data
EEPROM, enable the LVD (any voltage) and
wait for the internal voltage reference to
become stable. LVD interrupt requests may be
ignored. Once the LVD voltage reference is
stable, perform all EEPROM writes normally.
When writes have been completed, the LVD
may be disabled.

2. Configure the BOR as enabled (any voltage).
Select a threshold below VDD to allow normal
operation. If VDD is below the BOR threshold,
the device will be held in BOR Reset.



Best regards,

Ken Pergola

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

[Peter Anderson]

Thanks very much Ken.

I will try that when I next have spare time.

At the time, I don't think this errata was available.

P H Anderson

Post by Ken Pergola

...I had an odd problem with the PIC18F1320...

Hi Peter,
I remember your post about this. That was a strange
problem. Did you happen
to see the errata document regarding the EEPROM and
the LVD issue? See
2004 Microchip Technology Inc. DS80175B-page 1
Module: Data EEPROM
-------------------
When writing to the data EEPROM, the contents of
the data EEPROM memory may not be written as
expected.
Work around
-----------
1. Before beginning any writes to the data
EEPROM, enable the LVD (any voltage) and
wait for the internal voltage reference to
become stable. LVD interrupt requests may be
ignored. Once the LVD voltage reference is
stable, perform all EEPROM writes normally.
When writes have been completed, the LVD
may be disabled.
2. Configure the BOR as enabled (any voltage).
Select a threshold below VDD to allow normal
operation. If VDD is below the BOR threshold,
the device will be held in BOR Reset.
Best regards,
Ken Pergola
--
http://www.piclist.com hint: The list server can
filter out subtopics
(like ads or off topics) for you. See

http://www.piclist.com/#topics





__________________________________
Do you Yahoo!?
Win a $20,000 Career Makeover at Yahoo! HotJobs
http://hotjobs.sweepstakes.yahoo.com/careermakeover

--
http://www.piclist.com hint: The PICList is archived three different
ways. See http://www.piclist.com/#archives for details.

[Scott Thomas]

I am currently using an ICD2 and I *think* I can program the EEPROM with
it, but I haven't verified programmatically yet.

I am in the process of writing a test program that will demonstrate this
issue (or show me my mistake). I will post more when I know more.

Thanks for yours and everyone else's suggestions!

Post by Peter Anderson
Scott,
I had an odd problem with the PIC18F1320.
I was writing to EEPROM and then verifying. I
actually had to put it in a loop until it was
successful and there were times when it took over a
thousand writes.
I tried another style of programmer with much the same
result.
I went back to using the ICD2 which I had used to
debug the design and used it as a programmer and the
write was successful the first try everytime.

--
http://www.piclist.com hint: The PICList is archived three different
ways. See http://www.piclist.com/#archives for details.