Skip to content

Page 1

SIFT

Information Retrieval System

ND 210898

 _______________________________
|                               |
|           [Photo: Computer    |
|            screen displaying  |
|            information]       |
|_______________________________|

ND
Norsk Data

Scanned by Jonny Oddene for Sintran Data © 2010

Page 2

SIFT

Information Retrieval System

Introduction

SIFT is a general information retrieval system allowing searches in free text. Documents may be included with their full text or in the form of abstracts. Named fields may be defined in various ways, allowing the system also to handle forms-oriented documents. The database part is designed in such a way that even very large document collections may be searched and updated rapidly. The query language is general and powerful, but the simpler functions are very easy to use.

Features

  • Searching by boolean combinations of words (and, or, not)
  • Searching by word distance (adjacent words as a special case)
  • Searching for words in the same sentence or paragraph
  • Searching for truncated words
  • Searching on field values
  • Searching by classes (which incorporates non-boolean searching and relevance ranking)
  • Questions and results stored for later use
  • Search macros
  • Searching of several databases in parallel
  • Quick browsing through retrieved documents
  • Documents may be displayed or printed in user-defined formats
  • Highlighting of search terms
  • Retrieved documents may be sorted according to field values
  • Retrieved documents may be edited and updated online
  • User-defined fields and document layout
  • Parts of documents may be defined as non-searchable
  • Searching and display of certain document parts may be restricted
  • Access to databases may be restricted
  • Use of certain commands may be restricted

Applications

SIFT covers a very broad range of applications. It is suited for any large collection of "documents" (which may be real documents, reports, abstracts, descriptions of objects, etc.) that should be retrievable by their contents.

Some examples are:

  • Library services
  • Legal documents
  • Office archives
  • Catalogues
  • Publications
  • Newspapers

Product Description

Command language, help functions

SIFT is operated through a command language containing a fairly large number of commands and parameters. However, the inexperienced user needs to know only a few commands, and parameters are asked for only if they are mandatory or if the user has stated that (s)he wants parameter prompts by using the 'down arrow' key. The LIST COMMANDS command may be used to get a description of each command and its use. The 'HELP' key is accepted as a response to any prompt from the system, and will result in a detailed explanation of the answers or parameters that may be given at that point. Command or parameter input may be cancelled by using the 'HOME' or 'CANCEL' key. Some time-consuming operations may be stopped in the same way.

All meaningful keys on the NOTIS keyboard may be used, and the last command line may be edited. However, SIFT may also be used from tele-type or other non-NOTIS terminals, as well as from batch/mode jobs.

Database Administration

Any database is defined as 'owned' by the UE user who defined it. The 'owner' may control the access to the database by defining other users as 'friends' with various levels of authority for access to the database or to restricted parts of its documents.

Certain commands may also be defined as restricted, and in order to use such commands a user must be defined and sufficiently authorised in SIFT.

For each database the structure of references in the dictionary file may be defined, giving the possibility of saving disk space for databases which are not to be searched with advanced search operators.

'Stop words' (i.e. common words which have no relevance for searching, but which constitute a great part of the text) may be defined for each database in order to reduce the size of the dictionary file.

Page 3

SIFT Information Retrieval System

Retrieval

Retrieval is performed by giving the command FIND, followed by a word or string, using the SIFT query language.

The operators of the query language are:

  • Boolean operators: AND, OR, ANDNOT
  • Paragraph operators: PAR, NOTPAR
  • Sentence operators: SENT, NOTSENT
  • Word distance operator: /m,n/
  • Right-hand truncation:
  • Field operators: IN, EQ, LT, LE, GT, GE, BETWEEN, MAX, MIN, SAME
  • Class operator: CLASS

Parentheses may be used to achieve any degree of complexity. A question may use references to earlier questions. Macros may also be defined and referred.

Other commands used for retrieval are EXTEND, which extends the previous question, and LIST WORDS, which displays words starting with a specified stem. The command SELECT may be used to combine a subset of the displayed words to a query. RANK may be used to sort the result list according to assumed relevance, based on word frequencies.

Up to 25 databases can be searched in parallel.

Display of Documents

There are commands which allow easy navigation forwards or backwards through retrieved documents, or freely through a database. Navigation keys may be used in much the same way as in the NOTIS products. Documents may be directed to a printer with the PRINT key.

Formats for display and printing, consisting of selected fields, may be defined in various ways. The documents retrieved may also be sorted according to field values.

When the HIGHLIGHT mode is set, search words in the text will be shown in inverse video. When the FOCUS mode is set, only the context of the search words will be shown.

Requirements

SIFT runs only on the ND-500/5000 series of computers. SINTRAN III version K or later and User Environment version C or later are required. If documents are to be edited directly from SIFT, NOTIS-WP version N or later is also required.

The disk space required to establish a text database varies from 2.5 to 3.5 times the size of the original text, depending on the definitions and options used.

Documentation

  • SIFT Reference Manual (ND-60.278.1 EN)
  • SIFT Introduction (ND-60.283.1 EN)
  • Several document types may be defined for each database. A document type specifies attributes like:
    • Type of text storage (in internal SIFT files or as SINTRAN files)
    • The character sequences that separate:
      • Documents
      • Paragraphs
      • Sentences (normally '.')
      • Words (normally a space)
    • Document parts which are non-searchable
    • Document parts with restricted access
    • Which fields are to be included
    • How the fields are identified in the input documents

The fields themselves and their attributes are defined separately, and may be used in several document types. Attributes such as length, possible ways of searching and sorting, security level etc. may be defined.

It should be noted that the type and field-defining commands allow a wide variety of possible document layouts to be recognised by SIFT, but that SIFT does not produce the document layout. The documents must be produced (and possibly specifically prepared for SIFT use) by a text-editing system like NOTIS-WP or NOR-TEXT, or by a data-entry system. Any text in NOTIS standard format, NORTEXT format or pure ASCII (7-bit format) will be accepted, and all special characters and tabulator settings etc. will be retained.

Updating

There are commands to change the stop-word list, to add a new document (or a file of documents) to the database, to delete retrieved documents and to change a retrieved document. In the latter case, NOTIS-WP may (optionally) be called directly for editing of a retrieved document.

All these commands generate updating transactions. The updating process itself is activated by the UPDATE command. If the database files are defined as 'continuous', the database may still be used for searching while the updating process goes on.

If SIFT runs out of file space while performing a database update, it will halt and prompt the operator for appropriate action, and then continue normally. If a system breakdown or other serious problems should occur during a base update, the ROLLBACK command may be used to transfer the database back to the state it was in before the updating started.

Page 4

Norsk Data Contact Information

Corporate Headquarters

Address: - Olaf Helsets vei 5 - P.O. Box 25, Bogerud - 0601 Oslo 6 - Norway

Telephone: - Tel.: +47-2-629000 - Trunk: ..47-2-996500 - [illegible]

Telex: - 16474 nd n - 16435 nd n - 16598 nd n

Telefax: - 47-2-929266 (A)

Norway

Location Telephone Telefax
Oslo +47-2-629000 +47-2-629000 (A)
Bergen 47-5-342223 47-5-322633 (A)
Porsgrunn 47-35-99300
Sandnes 47-4-329660
Trondheim 47-7-291922

ND Comtec Head Office

Address: - P.O. Box 70, Bogerud - 0621 Oslo 6 - Norway

Telephone: - Tel.: +47-2-629000 - Trunk: ..47-2-661300

Telefax: - +47-2-629000 (A)

ND-Silviadata Sweden

Address: - Box 83 Sundsvall - Sweden

Telephone: - Tel.: +46-60-151510

Telex: - 20072 nd s

Locations: - Vaxjol, Tel.: +46-470-19500 - Stockholm, Tel.: +46-760-9840

Denmark

Norsk Data A/S: - Laurentsvej 13 - P.O. Box 122 - 2750 Ballerup

Telephone: - Tel.: +45-2-689065 - Telex: 35557 nd dk

Locations: - Copenhagen, Tel.: +45-2-609500 - Odense, Tel.: +45-9-151960 - Aarhus, Tel.: +45-6-613516 - Aalborg, Tel.: +45-9-873316

Finland

Address: - Box 45 - Oy Norsk Data Ab - Puukunniemekatu 4 - SF-20100 Turku 1

Telephone: - Tel.: +358-21-151280

Telex: - 1540 nd sf

West Germany

Norsk Data GmbH: - Thesdorferweg 101-3 - 6300 Bad Hamburg v.d.H.

Telephone: - Tel.: 06171-60034

Telefax: - 06171-2230 (A)

Locations: - West Berlin, Tel.: 030-8827317 - Hamburg, Tel.: 040-527002 - Hannover, Tel.: 0511-556065 - Stuttgart, Tel.: 0711-2561100 - Munich, Tel.: 089-985672 - Frankfurt, Tel.: 069-5000030

The Netherlands

Norsk Data Nederland B.V.: - Brouwserhof 50 - P.O. Box 90, 3340 AN Nieuwegein

Telephone: - Tel.: +31-3402-72400 - Telefax: +31-3402-72400

France

Norsk Data s.a.r.l.: - 4 rue Bréanton - Avenue du Jura, 01210 Ferney-Voltaire

Telephone: - Tel.: 33-450-56650

Switzerland

Norsk Data (Switzerland) S.A: - Chemin du Vallon 12 - CH-1009 Pully-Lausanne

Telephone: - Tel.: +21-210522

United Kingdom

Norsk Data Ltd: - Newbury,Berks RG16 8LU

Telephone: - Tel.: 044-635-3054

Ireland

Norsk Data Ireland Ltd: - Da Industrial Estate, Sunny Avenue - Dublin 3, Ireland

Telephone: - Tel.: 353-1-327444

USA

Norsk Data N.A., Inc.: - Westborough Office Park - 1800 W Park Drive Suite 150 - Westborough, MA 01581 USA

Telephone: - Tel.: 617-366-6565

ND International Operations

Address: - Olaf Helsets vei 5 - P.O. Box 70, Bogerud - 0621 Oslo 6, Norway

Telephone: - Tel.: +47-2-629000

Hong Kong

Norsk Data International: - Tel.: 852-688102

Iceland

Norsk Data Iceland Ltd: - Tel.: 35-562-13771

India

Norsk Data (India) Pty. Ltd.: - Tel.: 91-44-8149224

Pakistan

Norsk Data Pakistan (P) Ltd: - Tel.: 92-51-681446

Thailand

Norsk Data Thai Ltd: - Tel.: 66-2-236-8234

   ./+o+-.             :::.    +oooo+..
` ?>/  +oo+        .o+o/.     .+/:/++:+
'       +o://+..::::/o/::/+/.-+o/: '  \ 
           :  '       :.     . 

  ND
Norsk Data