I agree. I have already raised this with Kat...
louis

Hi,

A  request to change the run time of some rebuild scripts to lighten
the load has been entered with our operations department. We will kepp
you updated on this issue.

Thank you
Kat

the performance of the cvs server is worse than ever.

If cvs performance if related to this issue, I will raise priority to 
one, otherwise we have to set up a new issue.

times when site is slow have changed, but things get not better. When 
can we expect that the site  performs better ?

Hi,

I will check on other operational issues with our ops department and
the impact on performance.

Thank you
Kat

Hi,

Ops has reported there is a possibility of increasing the amount of 
RAM on the server, as well as other script run time changes to
increase performance. The earliest any of these options could be
considered due to personnel and hardware constraints is at least one
week from now.

Thank you
Kat 

I have been reported today by developers that a login last upto 20 
minutes, cvs log commands lasts upto 3 minutes. I think those values 
are not acceptable. 

Our operations department would like to try something to reduce the
load starting tonight.  
The idea is to disable cvsupd from 11PM-4AM, pacific time. This is
while the backups are running. The reasoning behind this is that
cvsupd is a major IO and CPU drain while it is running and they
believe it  to be causing  IO thrashing in the system. 
Disabling it for a few hours will test this theory and hopefully
restore some performance.

The downside to this plan is just that the anoncvs box will lag for a
few hours during this time.

Does this course of action sound acceptable?

Thank you
Kat

Tracked on internal issue pcn6008

Yes !

Hi Martin,

The crontab was changed last night so anoncvs was not  synced between
10PM-7AM pacific time.  Another script (slocate) which was causing
major load on the system, also competing with backups was changed from
running daily to 
weekly.

In the process of doing this,  it was found that significant drain on
the system still existed from a server in Germany running cvsup over
an ssh tunnel, every hour

sshd    32625 root    4u  IPv4   96935313               TCP
h66.sny.collab.net:s
sh->calzone.stardiv.de:769

The immediate suggestion from our ops department is for OpenOffice to
also discontinue cvsup during these hours (10PM-7am), as it is causing
drain on the system during the backup window. If the backups can
finish sooner because of less contention, we can shorten the window.
For example backups for other sites usually take only 3-4 hours.

An additional 1G of RAM is also available and can be installed on
Thursday, Oct. 25 at 21:30 PDT. This may also help to improve
performance of the site. The estimated downtime will be 15 minutes if
you choose to go through with this option.

Do either, or a combination of these proposals sound acceptable to
you?

Thank you
Kat

I have now reduced our cvsup times to 6.45, 12.45, 18.45 and 23.45 CEST.

I'm ok with a downtime Thursday, Oct. 25 at 21:30 PDT for 15 minutes (
I guess the additional RAM has already been tested ;) ).

The downtime slipped to approximately 22:00 PST due to a scheduling
conflict. The upgrade lasted for 31 minutes.

Thank you
Kat

Please see also #2070 (Cricket Data is not up-to-date).
Without this we have now feedback during our tests.

there seems to be a problem with cvsup. Every time cvsup is running it
tries to do an "SetAttrs" on a set of file (it seems to be every
file). The resulting file date then is January 1st 1970. This seems to
cause a high load on openoffice.org. We reduced the times of syncing
so load got lower. 
Is the anoncvs server working in a regular way ? if so what is the
command line for cvsup you use.
we're using Software and protocol version 16.1. then command line is:
cvsup -s -1 -d 10 -l LOCK.cvsup -P m -g -L1 supfile

We are upgrading the cvs client and server to 16.1 and will let you
know when it has been done.

Thank you
Kat

> We are upgrading the cvs client and server to 16.1 and ...
You're talking about cvsup (not cvs), aren't you?
Otherwise we had to veto!

Hi Stefan -

Sorry, I mis-typed in the last post. We are upgrading the cvsup on the
server to version 16_1e, and would like to request you do the same if
not on this version already. Ops feels this may also be causing some
of the performance problems we've been seeing recently, as this site 
explains in more detail.

http://people.freebsd.org/~jdp/s1g/

We have upgraded the cvsup client and it looks better now

We have also upgraded the client and server on our end. 

Kat

Performance reported back to normal on the call - seems cvsup was the
root cause. Closing the issue.

Kat

As agreed by Louis I will close these resolved fixed kat (support)-owned issues
now. If you have trouble with that, please re-open the issue.

As agreed by Louis I will close these resolved fixed kat (support)-owned issues
now. If you have trouble with that, please re-open the issue.