ACEDB User Group Newsletter - September 2001

If you want to have this newsletter mailed to you or you want to make comments/suggestions about the format/content then send an email to acedb@sanger.ac.uk.

General News

First HP, now IBM

Recently Hewlett-Packard lent us a whizzo test machine to try out acedb on and now IBM are "beating" a path to our door (well almost). Two people from IBM in the US visited us last month to discuss porting acedb to IBMs latest RISC machines and making a more long term commitment of supplying acedb binaries for IBM machines for download in addition to those we already supply.

Courtesy of Bradley Elkin of IBM (US) we now have a makefile for building acedb on AIX 4.3, the current release of IBMs version of the Unix operating sytem.

No more telnet access to Sanger

I received the below note from System Support at the Sanger Centre about a further tightening up of access from outside users. If this will affect you adversely please contact system support direct (ssg@sanger.ac.uk).

> Subject: Telnet Access to Sanger Systems.
> 
> 
> ** External Telnet access will be closed on Friday 12th October at 9am **
> 
> As part of the on going program to increase the security of the Sanger
> network we intend to close all remote access to Sanger systems via
> "Telnet"
> 
> Telnet access will not be closed internally although its use is
> discouraged.
> 
> Telnet does not use encryption and because of this it is possible for
> any data sent over the session, including user names and passwords to be
> captured.
> 
> We have installed, and been actively promoting the use of, SSH (Secure
> Shell) over the last 18 months. This provides a secure replacement for
> telnet with added functionality.
> 
> We have compiled a CD with SSH for all popular operating systems and these
> can be obtained from the System Support Group. If you require one please
> send a message to 'ssh-cd-request@sanger.ac.uk'
> 
> If you would like more information or have any questions please contact a
> member of the systems team.

New Features

Once more unto the process limits

Whenever you run a program on a Unix machine, the resources (memory, files open etc.) available to the program are controlled by a set of limits. You can view these using the built-in "limit" command with the csh shell:

> limit
cputime         unlimited
filesize        unlimited
datasize        262144 kbytes
stacksize       2048 kbytes
coredumpsize    unlimited
memoryuse       505664 kbytes
vmemoryuse      1048576 kbytes
descriptors     4096 
>

This can be a problem when running acedb if, by default, your process limits happen to be set low. In particular acedb requires reasonable datasize and stacksize limits, while the precise limits required depend on how big the database is and what sort of operations are being done, its likely that you will need at least 50MB datasize and 4MB stacksize. The simplest way to increase these limits is use the built-in "unlimit" command:

> unlimit data
> unlimit stack
>

It's hard to remember to do this however and accordingly, from this release, all acedb executables will automatically attempt to increase the limits for datasize and stacksize to the maximum permitted. If this is still very low (less than 50MB datasize and 4MB stacksize) you will be given a warning and asked whether you wish to continue. Server programs do not issue the warning but instead log the problem into database/serverlog.wrm within the database.

Reading in a set of dumpfiles

A long time ago, in a galaxy far, far away....Roger Horton said rather wistfully "Wouldn't it be nice if you could just click on something and read in a set of acedb dump files in one go.". Well, now you can.

There are two interfaces:

xace
From the "Admin.." menu on the main window, select "Read Dump Files". The file chooser window will be displayed and will by default show the directory you were in when you started xace so if this is where the dump files are, you will be able to select one immediately.

To load a set of dumpfiles, just choose the first file in the set which will have a name something like "dump_2001-09-05_A.1.ace", you will then be shown a message window which will identify the dump set to be read in and ask you if you want to procede. If you say "Yes", the dumpset will be read in and a list of files read in/parse results for the files will be displayed.

If errors are encountered in a dump file then processing of the dump set will stop at that file. If you decide at that point you want to just abort the whole dump reading then you should quit acedb without saving and start again.

tace
Use the new "dumpread" command to read in a dumpset: "dumpread <:dump_file_name>". As with xace, you will be asked to confirm that you want to read in the dumpset, and once finished the list of files read/parse results will be displayed.

Dumping CDS's from a keyset

The "Export.." menu in the keyset window now contains two extra options to dump just the CDS in FASTA format as either DNA or protein for the keyset objects.

Objects that do not contain the "CDS" tag and associated data will not be dumped.

saceclient, scripts, userids and passwords

Following a discussion on the acedb newsgroup about running saceclient from a script the following changes have been made to saceclient:

Fixing of the bug means that it's now possible to run saceclient via a pipe from a script, pushing through the pipe the userid and password when required. This method is slightly more complex to program but is more secure than using the command line arguments (see below).

The command line arguments are easier to use in that saceclient can be started and the connection made to the database in a single line. You should be aware however that this option causes a security hole, the problem is that its possible for anyone to log on to the machine where saceclient is running and via the ps command find out your password:

> ps -f -u edgrif
UID         PID   PPID    C STIME    TTY             TIME CMD
edgrif      405      1  0.0 09:10:56 ??           0:04.88 /usr/bin/X11/xclock -geometry 195x28-0-0 -digital -bg white -fg black -font *h
                                          .
                                          .
                                          .
                                          .
edgrif     7113   7038  0.0 13:50:29 ttypb        0:00.41 saceclient -userid edgrif -passwd wallychops -port 20001 griffin
>

Oops, damn, now I've given my favourite password away...sigh.

This may not matter to you if are running on a machine that is not accessible to others but you should be aware of this potential problem.

Read only databases and saceserver

A common requirement for giving general, wide spread access to databases is to be able to make the database "read only" so that it cannot be either accidentally or maliciously damaged. On a Unix system this means making the database directory and all its files/subdirectories readonly to the server and perhaps not accessible to other users at all.

Assuming that you will run the saceserver as userid johnboy and you want only johnboy to have access to the database, and that johnboy is the owner of all files/directories in the database.

Prevent other users from having any access to the database:

> chmod -R go-rwx <your_database>

Prevent johnboy from having write access to the database in any way:

> chmod -R u-w <your_database>

You must then uncomment the READONLY_DATABASE option in the wspec/serverconfig.wrm file for the database to tell the server that the database is intended to be read only. If you don't do this the server will detect that it cannot write to its log file, issue an error message and exit.

Although this arrangement is very secure it does mean that none of the normal logging will take place. Its possible to continue logging but keep the rest of the database read only as well. Follow the above steps but with two important differences:

Make sure that the server can write to the server log file:

> chmod u+w <your_database/database/serverlog.wrm>

Make sure that the READONLY_DATABASE option in the wspec/serverconfig.wrm for the database is commented out, this will tell the server that it should try to write to its logfile.

This will enable the server to log connections etc. with reasonable security.

Bugs Fixed

File chooser bug

The file chooser sometimes displayed the parent directory of the one actually required (usually the PWD), this is now fixed, the file chooser should always now show the requested directory.

Fmap scrolling bugs

Two bugs from when horizontal scrolling was introduced into fmap have been fixed:

Reverse strand display in Blixem

A small buglet in blixem broke reverse-strand display, this is now fixed.

Developers Corner

If you wish to make comments/suggestions about any of the below, please mail them to acedb@sanger.ac.uk

Small additions to filsubs

The new filGetDirname() function will return just the directory portion of a path, this complements existing functions to return filename and extension. filCheck() now includes a test for the existence of a file, n.b. you may still not be able to read the file but you can see if it is there !

GNU getopt code added

The GNU getopt code has been added, this gives us support for single letter and long form command line arguments, the calls in this small package are summarised in wdoc/getopt.html. This is the first part of a two stage plan to standardise our handling of command line options. The aim being a) to bring our handling of command line arguments into line with standard Unix practice, and b) to enable us as developers to add new command line options simply by updating a table of such options and then making a single call later in the code to find out whether an option/value has been specified.

aceOutPrint() bug again

When the aceio package was written the interface for aceOut was incomplete in that it included an aceOutPrint() call which should have been named aceOutPrintf(). This has led to the below usage of aceOutPrint():

       char *some_string ;

       aceOutPrint(some string) ;

Of course there is a problem if some_string contains the printf formatting character "%", this has tended to lead to incorrect output rather than crashes. Some time in the next month or so documentation will be added and a new "plain string" call added to the aceout package to make the distinction between formatted and plain text output.

September monthly build now available.

You can pick up the monthly builds from:

Sanger users
~acedb/RELEASE.DEVELOPMENT
External users
http://www.acedb.org/Software/Downloads/monthly.shtml

Next User Group Meeting - D319, 2.30pm, Thursday, 18th September 2001



Ed Griffiths <edgrif@sanger.ac.uk>
Last modified: Mon Nov 12 08:30:02 GMT 2001