Science and Technology news... :: View topic

Heather B · Guest

Hi there,

I have a question about the age of work units. I run a caching program that
caches 15 work units. II>m wondering how much time I have to get those
downloaded work units processed and get the results back to the SETI server
before they are considered lost, and the same work units are given to a
different user to process? I don>t want to cache too many, but on the other
hand, I want to be able to sustain processing over long periods without an
internet connection.

Thanks,
Heather

Terry Groff · Guest

Hi Heather,

WU>s don>t expire. You will always get credit for WU>s
regardless of how long you>ve had them

"Heather B" <heather.b@attglobal.net> wrote in message
news:bh98s6$ds5$1@bob.news.rcn.net...
[quote]
Hi there,

I have a question about the age of work units. I run a caching
program that
caches 15 work units. II>m wondering how much time I have to
get those
downloaded work units processed and get the results back to the
SETI server
before they are considered lost, and the same work units are
given to a
different user to process? I don>t want to cache too many, but
on the other
hand, I want to be able to sustain processing over long periods
without an
internet connection.

Thanks,
Heather
[/quote]

Hi Heather,

WU>s don>t expire. You will always get credit for WU>s
regardless of how long you>ve had them. I personally cache 21
WU>s

Terry

Jim Kent · Guest

On Tue, 12 Aug 2003 11:08:06 +0200, "FalconFly" <falconfly@ewetel.net>
wrote:

[quote]You have about 48 hours (max.) to return them, in order to get the Results into the
scientific Database.

Unless the Client expires, you>ll indeed always get credit in the Stats, but cacheing too
long makes the Results useless
Absolute max. I would stive for is less than a week, e.g. 3-4 days.
[/quote]
Hold on here.... 48 hours before becoming invalid? Then how can you
simultaneously suggest a three or four day cache? That seems
logically inconsistent, because every one of those units would be
worthless.

Your assertion about the 48 hour liimit to get into the scientific
database is a bit hard to believe, because a lot of people are still
running machines that take that long to do a single unit. If these
are all invalid that effort is utterly pointless and a waste of their
time, electricity, and all the work unit management effort at SETI
that supports these users.

Maggot · Guest

In article <ll5ijv0l79aqb1stjl0on0oi703pktsh7n@4ax.com>,
j.g.kent@markusplace.com says...
[quote]On Tue, 12 Aug 2003 11:08:06 +0200, "FalconFly" <falconfly@ewetel.net
wrote:

You have about 48 hours (max.) to return them, in order to get the Results into the
scientific Database.

Unless the Client expires, you>ll indeed always get credit in the Stats, but cacheing too
long makes the Results useless
Absolute max. I would stive for is less than a week, e.g. 3-4 days.

Hold on here.... 48 hours before becoming invalid? Then how can you
simultaneously suggest a three or four day cache? That seems
logically inconsistent, because every one of those units would be
worthless.

Your assertion about the 48 hour liimit to get into the scientific
database is a bit hard to believe, because a lot of people are still
running machines that take that long to do a single unit. If these
are all invalid that effort is utterly pointless and a waste of their
time, electricity, and all the work unit management effort at SETI
that supports these users.

Jim, here>s the FAQ from SETI:[/quote]

http://setiathome.berkeley.edu/faq.html#q1.17
"What is an "optimal" cache size to avoid returning obsolete results?
Caching is now more popular than ever. (See our add-ons page for some of the
more popular caching programs.) In general, a 1-month cache isn>t bad in
terms of redundant results, but a 7-day cache is much better. An "optimal"
cache (minimizing the likelihood that a result will have already passed
integrity testing before you return it), however, shouldn>t hold more than a
2 days of workunits. In the past, 1 month was pretty safe, but the
ramifications of Moore>s Law have made the cache window much smaller. Keep in
mind, of course, that excepting the above case, redundancy is extremely
important for testing the integrity of our data. Interestingly, even without
cacheing, users on average will receive a duplicate workunit (one they>ve
processed before) about once every 500 times. "

Mike
--
"All my friends can>t be wrong!"
Remember, stupidity runs in herds.

Martin · Guest

Jim Kent wrote:
[quote]On Tue, 12 Aug 2003 11:08:06 +0200, "FalconFly"
falconfly@ewetel.net> wrote:
You have about 48 hours (max.) to return them, in order to get the
Results into the scientific Database.

[...]

Your assertion about the 48 hour liimit to get into the scientific
database is a bit hard to believe, because a lot of people are still
running machines that take that long to do a single unit. If these
are all invalid that effort is utterly pointless and a waste of their
time, electricity, and all the work unit management effort at SETI
that supports these users.
[/quote]

Trying to avoid getting lost in too much detail...
(But failing badly! (:-O))

The conclusion is:

In short, you gamble on how quickly others return their WUs compared to
yourself, and how large a WU pool size s@h maintain. Gamble on too long
to get your WU result back to the server and that extra result for the
WU is discarded. You still get credited with one more point for your WUs
completed statistic.

The s@h servers are very unlikely to be offline for more than 2 days
(some disaster over a weekend to be fixed Monday morning), so it is
pointless having more than 2 days of cache.

(And BIG caches make the situation harder for the s@h servers (:-))

Until s@h can be a little more open about utilisation, if you need a
larger cache due to being offline, perhaps you should look at other
distributed computing projects that can tolerate long delays for the
results and better utilise your power.

In further explanation, my 'best guess' on s@h>s current operation:

(Look up the earlier threads on cache sizes...)

The s@h servers send out in one gulp 3 or 4 (or 5 or more) copies of a
particular WU.

Some time later, an 'offline' server job is run which checks for a
minimum of 3 results for each WU in the 'server pool'. If that WU has
got 3 or more results, then it is removed from the WU 'pool', otherwise
it is left there for further copies of it to be sent to more users.

Now, in the real world, some users return their results in less than two
hours after receiving the WU from the s@h server. Others likely have
slower machines or delay things further by the size of their WU caching
and WU results caching.

Your WU result, to be scientifically useful, is in a race to get back to
the s@h server before that WU is deleted from the s@h server WU pool.

The time delay before a WU is deleted from the pool is a minimum of how
long it takes for 3 results to be returned. The maximum time is a
combination of the size of the WU pool and how quickly the WUs are
replaced by new WUs from the tape splitters. The number of WUs swallowed
up by user caches also complicates this story...

There have been a number of (old) authorative posts to say that a WU
stays in the pool for less than one month. With the increase in active
users, and faster machines, and the use of caching, that time may now be
less if the s@h tape splitters can supply new WUs quickly enough.

s@h list on their site that the average WU processing time is 8hr 57min
48.9sec over the last 24 hours.

See:
http://setiathome.ssl.berkeley.edu/totals.html

Unfortunately, there are no direct stats indicating the churn rate on
the WU pool, or the pool size, or the proportion of those WUs
distributed to users. Hence, the useful lifetime for a WU is guesswork.

Some information is sampled by:
http://www.roving-mouse.com/setiathome/

I run with a very small cache to try to keep the results useful. Another
project gets my spare cycles if s@h can>t keep their servers online.

Keep crunchin'
Martin

--
----------
- Martin -
- 53N 1W -
----------

Jim Kent · Guest

On Tue, 12 Aug 2003 17:14:41 GMT, Maggot <Maggot@nospam.invalid>
wrote:

[quote]Jim, here>s the FAQ from SETI:
[/quote]
Argh. I had looked though the whole FAQ and managed to miss this
paragraph. Thanks.

My point is still valid. Anyone running a machine that takes more
than two days to process a WU should shut it down. It>s not doing
anything useful. And no one should be configuring a cache holding
more than two days worth of WUs because that>s equally pointless.

FalconFly · Guest

Well, my wording wasn>t very clear :

After 48hrs, 'most' Results return back to the Server.
This is just an average figure, and there are Results that return in less than 3hrs, as
well as some beyond 48hrs of course.

It>s just, that the average possibility to have the Results getting their "Verified" Tag
is increasing, along an increasing delay in return time.

So there>s no "hard line", just an average figure of 24-48hrs.

It means that anything beyond that starts to scale down in the possiblity to get fully
accredited to the Database as a contributing Result to the Verified status (which is of
course the thing to aim for).

Going beyond that (e.g. 3-4 days) in case of having no other way to return the Workunits
quicker, simply reduces their chance, but does not render them absolutely useless.

See it as a compromise, based on the User>s Priorities :
If he/she wants to score a persistent 100% in getting the Results definitely into the
Database, the 24-48hrs max. will be what to stive for, and consequently to shutdown
anything that is prone to exceed this time.

But if, let>s say, a 75-90% are 'good enough' due to circumstances, some might accept this
as a useable compromise between desired ideal case, and limiting factors that work against
it.

And even with those figures, the Administration stated that isolated Workunits actually
never make it back to the Server (for a multitude of reasons). With ~1.2 Million being
sent out per day, the statistical chance is low, but obvious.

Since there>s no detailled statistics on that matter, this is all based on the official
Posting several month ago, but I reckon this is a useful estimation.
IMHO, totally pointless are caches approaching or exceeding ~7 days, since the odds of
'scoring a hit' is just getting extremely low.

I hope that explains it a bit better, why there>s no 'hard boundary' but rather large
fields of grey that seemingly blend between Ideal case, sub-optimal, undesirable and
totally useless ;)

Greetings
FalconFly

Webmaster
http://www.falconfly.de
3dfx Archive
====================
Forum : http://www.falconfly-central.de

Nick M V Salmon · Guest

"helmsman" <helmsman@mindspring.com> wrote
[quote]"Nick M V Salmon" <spam_dump@btinternet.com> wrote:
[/quote]
[....]

[quote]We>re currently processing circa 1,300,000 WUs/day, ie. close to 6 x
realtime..! No wonder they say "Anyone running a machine that takes
more
than two days to process a WU should shut it down."..! IMO it>s a
pointless
waste of electricity..................

Irrelevant, We purchase the electricity and it>s a neet screen saver.
And on top of everything, my cats like to watch it :')
[/quote]
<LOL>

[UK]_Nick...

Stratcat® · Guest

"Jim Kent" <j.g.kent@markusplace.com> wrote in message
news:ll5ijv0l79aqb1stjl0on0oi703pktsh7n@4ax.com...
[quote]On Tue, 12 Aug 2003 11:08:06 +0200, "FalconFly" <falconfly@ewetel.net
wrote:

You have about 48 hours (max.) to return them, in order to get the
Results into the
scientific Database.

Unless the Client expires, you>ll indeed always get credit in the Stats,
but cacheing too
long makes the Results useless
Absolute max. I would stive for is less than a week, e.g. 3-4 days.

Hold on here.... 48 hours before becoming invalid? Then how can you
simultaneously suggest a three or four day cache? That seems
logically inconsistent, because every one of those units would be
worthless.

Your assertion about the 48 hour liimit to get into the scientific
database is a bit hard to believe, because a lot of people are still
running machines that take that long to do a single unit. If these
are all invalid that effort is utterly pointless and a waste of their
time, electricity, and all the work unit management effort at SETI
that supports these users.

[/quote]
Yeah, but the guys with 2.5hr/WU machines will have already sent the 2 - 3
confirming WU>s that will get credit for the science, long before your unit
comes in.

All you>ll be doing is building stats, and adding a small amount of
additional corroboration to the earlier corroborating units. Not saying
additional corroboration is a bad thing, per se, but the major 'science'
will have already been 'done'.
--
Strat®

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.507 / Virus Database: 304 - Release Date: 8/4/2003

Robi · Guest

Stratcat® wrote:
[quote]Jim Kent wrote:
FalconFly wrote:

You have about 48 hours (max.) to return them, in order to get the
Results into the scientific Database.
[/quote]
and opposed to this, FalconFly continues:

[quote]Unless the Client expires, you>ll indeed always get credit in the
Stats, but cacheing too long makes the Results useless
Absolute max. I would stive for is less than a week, e.g. 3-4 days.
[/quote]
which contradicts the first statement.
Then again if a slow computer, crunching on 1 WU for 200+ hours returns
a correct WU, whereas 2 or 3 extremely OC PCs crunching a WU in less
than 1 hour return garbage, then I would assume that the result that took
8+ days is more useful than the fast results which are useless...

OTOH I have a cache of 1 week, which allows me to continue crunching away
in case of a before seen outage...

[quote]Hold on here.... 48 hours before becoming invalid? Then how can you
simultaneously suggest a three or four day cache? That seems
logically inconsistent, because every one of those units would be
worthless.

Your assertion about the 48 hour limit to get into the scientific
database is a bit hard to believe, because a lot of people are still
running machines that take that long to do a single unit. If these
are all invalid that effort is utterly pointless and a waste of their
time, electricity, and all the work unit management effort at SETI
that supports these users.
[/quote]
this supports my statement of the PC crunching a WU in over 200 hours ;o)

[quote]Yeah, but the guys with 2.5hr/WU machines will have already sent the 2 - 3
confirming WU>s that will get credit for the science, long before your unit
comes in.
[/quote]
and who knows, maybe those are precisely the useless results because they
have been analyzed on extremely OCed machines and return inconsistent
results.

[quote]All you>ll be doing is building stats, and adding a small amount of
additional corroboration to the earlier corroborating units. Not saying
additional corroboration is a bad thing, per se, but the major 'science'
will have already been 'done'.
[/quote]
Yogi Berra once said: "It ain>t over 'till it>s over"
and with elections it>s "nothing is final until all the votes are in"
and in US elections (FL) it adds: "... and the mistakes are accounted for"

so if 5 5 WUs were sent and all 5 results are inconsistent, then further
WUs need to be sent until consistent results are received.
That>s the science, and not who gets their junk back faster.

Just my 2¢

--
Robi
(2.7#@ 2.62 yrs)

Robi · Guest

Robi wrote:
[...]
[quote]so if 5 5 WUs were sent ...
^^^[/quote]
oops! a hiccough!
should read:
[quote]so if 5 WUs were sent ...
[/quote]
sorry about that

--
Robi
(2.7#@ 2.62 yrs)

Stratcat® · Guest

"Robi" <r_buecheler@remove.yahoo.com> wrote in message
news:vjm5hs57of9k37@corp.supernews.com...
[quote]Robi wrote:
[...]
so if 5 5 WUs were sent ...
^^^
oops! a hiccough!
should read:
so if 5 WUs were sent ...

sorry about that

[/quote]
Hey, what>s a late nite typo among friends??? <g>

But if you really want an example an embarrassing screw-up, you should see
what a guy did in another n/g. It was the weekend, and he met a girl in a
bar. He got her email addy, went home drunk when the bar closed, and at 3
in-the-morning composed a long drunken email tellin' the girl all kinds of
stuff about how intelligent she was, and he enjoyed their conversation,
etc., etc., finally ending by telling her he hoped they could become better
friends and get together again, or some such B.S.

Then...He accidentally posted it to the newsgroup where he was well known,
instead of emailing her!!! Loads of laughs and humility for days!
--
Strat®

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.507 / Virus Database: 304 - Release Date: 8/4/2003

Martin · Guest

Robi wrote:
[...]
[quote]OTOH I have a cache of 1 week, which allows me to continue crunching away
in case of a before seen outage...
[/quote]
No significant general connection problems have been seen since s@h got
their 100Mb/s Cogent link commisioned. The only interruptions now are
very short duration when they rearrange the servers.

I doubt we>ll see again the sort of connection problems that were
suffered leading up to and during the transfer over to Cogent>s
generously donated link.

Also, a one week cache is in the guestimated grey area where some or all
of your WU results may well become stale.

Keep crunchin'
Martin

--
----------
- Martin -
- 53N 1W -
----------

Jim Kent · Guest

On Wed, 13 Aug 2003 23:53:59 -0500, "Robi"
<r_buecheler@remove.yahoo.com> wrote:

[quote]extremely OC PCs crunching a WU in less
than 1 hour return garbage
[/quote]
Is there some factual basis for this supposition?

Robi · Guest

Jim Kent wrote:
[quote]Robi wrote:

extremely OC PCs crunching a WU in less
than 1 hour return garbage

Is there some factual basis for this supposition?
[/quote]
"if" Jim, *if*.
[quote]Then again if a slow computer, crunching on 1 WU for 200+ hours returns
a correct WU, whereas 2 or 3 extremely OC PCs crunching a WU in less
than 1 hour return garbage, then I would assume that the result that took
8+ days is more useful than the fast results which are useless...
[/quote]
And yes.
there have been "extremely" OCed machines returning nothing but garbage.
That was another reason why Roelof used to check the results from the
benchmark WU.

--
Robi
(2.7#@ 2.62 yrs)


www.GetXFactor.com Leading Technology, Science, Agriculture News and information	Part of the Identityscape.com network... getxfactor.com jmoodmusic.com smartbusinesschoices.com mintdepot.com lowfaresalways.com evangelicalview.com shoppingpodder.com soproudlywehail.com webnews.ws currenthumor.com

Science and Technology news... Forum Index -> Astrology - SETI Forum	Goto page 1, 2 Next
Page 1 of 2