Overview
Data processing
Summary
Web interface
Summary
A brief guide to the ACIS source files
A detailed guide to the ACIS source files
Traditional meta files
Documentation
Installation and configuration
Presenters and other presentation-related files
General
Users’ screens
Person-profile editing screens
New user (initial registration) screens
Administrative screens
Email generation
Other
ACIS:: hierarchy
ACIS::Web — Web interface of ACIS
Research Profile underlying modules
APU
Citations
Other ACIS:: modules
Web::App — the web application framework
ACIS::Data::DumpXML
ARDB — the data-processing
SQL helper
Other
Internally, ACIS consists of two main subsystems and a number of smaller tools. These two main parts are very different and do not know much about each other; they work or can work pretty independently. First there is the data processing subsystem. Second there is the web interface subsystem. The glue that makes them perform together as a single application is the predefined and coordinated configuration, some shared modules and a bunch of scripts.
The data processing subsystem processes input metadata. It puts the data into database tables for the web interface to use. The web interface subsystem handles user requests arriving through the web server. Sometimes it creates data files, which data processing subsystem will process. The system is complex, but consists of simple parts; each part has distinct responsibilities.
The data processing subsystem, in its turn, consists of two loosely coupled parts. The first part monitors data files, tracks their modifications, keeps records about data which is found in those files and filters out data records, which have bad identifiers. For historical reasons, it is called RePEc-Index, RI for short.
The RePEc-Index is built around a simple idea that metadata comes from
collections. Each collection has a name (identifier), a
type and is stored in data files somewhere in the
filesystem. Metadata is only useful when someone processes it for
something. So each collection may have further processing
defined for it. All these things are specified in the collections
configuration. ACIS creates this configuration for RePEc-Index
in the file RI/collections.
In a collection, each file may contain zero, one or many data records. Each data record must have a unique identifier. If two or more records in a collection have the same identifier, there is a conflict. RePEc-Index excludes the conflicting records from further processing, until there’s only one record left with a particular id. When a data record is successfully read from a data file, RePEc-Index does its checks and, if everything is ok, executes further processing for it.
That is when the second part of data processing comes into play. The second part is ARDB (abbreviation from Abstract RePEc DataBase, again with historical reasons). RI sends ARDB a record object, and ARDB processes it. To process a record may mean to extract certain pieces of information from it, run arbitrary perl code on it, store it in a database table. Similarly, RePEc-Index calls ARDB when a record disappears (or when RI discovers an identifier conflict). ARDB then cleans up the database. It removes the data which originated from that record.
ARDB’s work is governed by an elaborate configuration.
The configuration is stored in
(configuration.xml in the ACIS home directory. The
configuration specifies what ARDB has to do when
processing a record of a specific type. Possible
kinds of actions: store some of the record’s data into a
database table, execute certain perl code, extract a
relationship of this record to some other record. When
cleaning up a record, ARDB can remove database table records
or call perl code.
ARDB’s configuration also defines database tables. Those may be tables, which other parts of the configuration refer to, but they don’t have to. Either way, ARDB can create those tables for you; there is a script for that (bin/create_tables). ACIS relies on this capability of ARDB to create tables which are needed.
Now, let me again outline the workflow of data processing. When RePEc-Index finds a new or changed data record, and this record has a valid unique identifier, it asks ARDB to process it. When RePEc-Index finds that a record has been removed from a data file, it asks ARDB to remove the record’s data. Also, when there is an id conflict, RePEc-Index will request ARDB to clean up the conflicting record’s data. The connection between RePEc-Index and ARDB is not intrinsic; it is just in the way of how ACIS configures RI to process the metadata.
ACIS consists of data processing part and web-interface part. There are also some other smaller tools, which serve as glue.
Data processing system is RePEc-Index and ARDB. RePEc-Index monitors data files for changes and checks records for id uniqueness.
ARDB stores data in database tables and cleans it up when necessary. The work it does is specified in an elaborate configuration.
In ACIS, RePEc-Index is configured to call ARDB for the actual data processing.
The web interface part of ACIS is:
Perl code: CGI script + framework modules + ACIS-specific modules + helper scripts
a set of XSLT files called presenters
web application configuration, screens.xml
There are two main parts in the set of perl modules that
make ACIS web interface work. First is the framework. It
contains general code for analysing a user’s request,
deciding how to treat that request and generating a
response. It helps ACIS to store and load user sessions and
does a bunch of other little things for a web application.
That’s why I call it the web application framework. It’s
core is in the Web::App module.
The second part contains the specifics of ACIS. There lives
code about ACIS user accounts, searching for research items
and research institutions, creating and updating a personal
profile and so on. This part is rooted at the
ACIS::Web module. The framework (ie. the previous
part) makes extending and debugging this part much easier by
providing tools and a common environment.
These two parts are tied together by:
inheritance — ACIS::Web class inherits from
Web::App and extends it in ACIS-specific ways.
screens.xml — the application configuration.
The application configuration, the screens.xml file,
is built around the notion of a screen. Screens are basic
units of the web interface; they handle incoming requests
and generate responses. Each screen represents a certain
piece of web-accessible functionality of the system.
Web::App looks at addresses (URLs) of the requests that
arrive. For each request it decides which screen it is for.
Each screen definition in screens.xml configuration
specifies which modules and which functions will
Web::App invoke for it. If a request came for an
unknown (unspecified) screen, Web::App will generate
a 404 error and display a “Sorry” screen.
Another thing that the screens configuration brings into the mix is presenters. For each screen it defines an XSLT file, which is used to generate a response page. This means the application presentation is strictly separate from the main application logic.
All this separation between web-application general and ACIS-specific, between presentation and application logic works for flexibility and extensibility of the system. It is an attempt to follow the “separation of concerns” principle.
At the same time, the CGI script in ACIS is so simple that I
could have left it unmentioned without harm. Basically it
creates an object and calls a method on it (or two) and there it
ends. The other web-interface-related scripts mostly act in
a similar way: they involve a certain part of the
ACIS::Web and it does the rest.
ACIS is built on top of a generic web-application framework.
ACIS::Web module and many other modules in
the ACIS::* hierarchy encapsulate all what’s
essentially ACIS-specific.
All of the user interface is coded in XSLT stylesheets (templates).
Application configuration (screens.xml) relates
different pieces together, uniting all of the above into a
single solid application.
Main parts and some key files
This is an annotated list of most files of an ACIS distribution.
COPYINGMANIFESTMANIFEST.SKIPMakefile.PLREADMETODOdoc/make.linked.plxsltproc in PATH.
doc/make.simple.pldoc/check-filelist.pldoc/internal.text and in MANIFEST. Obsolete.
doc/style.cssdoc/xslt/010.xsldoc/xslt/020.xsldoc/xslt/030.xsldoc/index.htmldoc/adm.htmldoc/apache-conf.htmldoc/conf.htmldoc/install.htmldoc/bdb-private.htmldoc/overview.htmldoc/researchprofile.htmldoc/apu.htmldoc/cooperate.htmldoc/eprints.htmldoc/eprints-install.htmldoc/overview.htmldoc/citations.htmldoc/db.htmldoc/daemon.htmldoc/internal.htmldoc/*.texthome/bin/conf.plmain.conf and creates thisconf.sh,
ardb.conf, acis.conf
home/bin/ridhome/bin/setuphome/bin/setup.cgi_frontendhome/bin/setup.logs-browsinghome/bin/setup.ri_collectionshome/bin/setup.ri_local_setup.pmhome/bin/setup.sid_local.pmhome/bin/templates/apu.plhome/bin/templates/clean-up.plhome/bin/templates/create_tables.plhome/bin/templates/upgrade_to_*.plhome/bin/templates/*.plhome/configuration.xmlhome/contributions.conf.xmlmain.conf.eghome/screens.xmlinstall.shhome/presentation/default/global.xslshow-status
template. home/presentation/default/page.xslpage template) and a special markup, used all over
the place. Directly or indirectly it is used by every
HTML-page presenter of ACIS.home/presentation/default/forms.xslhome/presentation/default/page-universal.xslappropriate-page and
appropriate-page-soft template, which display
new-user-page to new users and user-page
to returning users.home/presentation/default/errors.xmlerror() method in
Web::App. home/presentation/default/messages.xmlmessage()
method in Web::App. home/presentation/default/fields.xmlshow-status template.home/presentation/default/fields-institution.xmlfields.xml (see previous item)
in case of the new-institution screen.home/presentation/default/index.xslhome/presentation/default/misc/login-pass.xslhome/presentation/default/misc/login.xslhome/presentation/default/misc/sorry.xslhome/presentation/default/misc/local-document.xslACIS::Web::Site.home/presentation/default/phrase.xml<phrase ref=’…’/> markup elsewhere in
templates. Such a phrase element will be replaced with
content from this file or its installation-local equivalent
{HOME}/presentation/default/phrase-local.xml. home/presentation/default/script/main.jshome/presentation/default/script/jquery.jshome/presentation/default/style/brownish.css.addhome/presentation/default/style/ie-font-sizes.csshome/presentation/default/style/main.csshome/presentation/default/user/page.xsluser-page template for all users’ screens.home/presentation/default/user/welcome.xslhome/presentation/default/user/settings.xslsettings screen.home/presentation/default/user/good-bye.xslhome/presentation/default/user/unregister.xslhome/presentation/default/user/account-deleted.xslhome/presentation/default/person/page.xslhome/presentation/default/person/affiliations-common.xslhome/presentation/default/person/affiliations-ir-guide.xslhome/presentation/default/person/affiliations-search.xslhome/presentation/default/person/affiliations.xslhome/presentation/default/person/affiliations/new-institution.xslhome/presentation/default/person/contact.xslhome/presentation/default/person/name.xslhome/presentation/default/person/interests.xslhome/presentation/default/person/photo.xslhome/presentation/default/person/profile-overview.xslhome/presentation/default/person/profile-static.xslhome/presentation/default/person/profile-show.xslprofile-overview.xsl and profile-static.xsl
home/presentation/default/person/research/*.xslhome/presentation/default/citations/*.xslhome/presentation/default/person/generic.xslhome/presentation/default/new-user/page.xslnew-user-page template.home/presentation/default/new-user/initial.xslhome/presentation/default/new-user/additional.xslhome/presentation/default/new-user/complete.xslhome/presentation/default/new-user/confirm.xslhome/presentation/default/adm/events-decode.xsl/adm/events/decode screen
home/presentation/default/adm/events-raw.xsl/adm/events/raw screen
home/presentation/default/adm/events.xslhome/presentation/default/adm/index.xsl/adm screen
home/presentation/default/adm/pass.xsl/adm/… screens
home/presentation/default/adm/search-res-doc.xslhome/presentation/default/adm/search-res-rec.xslhome/presentation/default/adm/search-res-usr.xslhome/presentation/default/adm/search.xslhome/presentation/default/adm/session-deleted.xslhome/presentation/default/adm/session.xslhome/presentation/default/adm/sessions.xslhome/presentation/default/adm/sql.xslhome/presentation/default/email/general.xslhome/presentation/default/email/*.xslhome/presentation/default/misc/forgotten-password.xslhome/presentation/default/widgets.xslhome/presentation/default/indent.xslhome/presentation/default/export/amf-person.xslhome/presentation/default/export/redif.xslhome/presentation/default/misc/time.xslhome/presentation/default/misc/time-test-data.xmlhome/presentation/default/stub.xsllib/ACIS/Web.pmlib/ACIS/Web/Admin.pmlib/ACIS/Web/Admin/Events.pm/adm/events/decode and /adm/events/raw.
lib/ACIS/Web/Affiliations.pmlib/ACIS/Web/Background.pmACIS::Web::Background module. Its all about forking
a process and keeping track of the forked threads.
lib/ACIS/Web/CGI/Untaint/latinname.pmlib/ACIS/Web/CGI/Untaint/name.pmlib/ACIS/Web/CGI/Untaint/password.pmlib/ACIS/Web/CGI/Untaint/simpleemail.pmlib/ACIS/Web/CGI/Untaint/url.pmlib/ACIS/Web/Config.pmlib/ACIS/Web/Contributions.pmlib/ACIS/Web/Export.pmlib/ACIS/Web/Import.pmlib/ACIS/Web/NewUser.pmlib/ACIS/Web/Person.pmlib/ACIS/Web/SaveProfile.pmlib/ACIS/Web/Services.pmlib/ACIS/Web/Session.pmlib/ACIS/Web/Session/SMagic.pmlib/ACIS/Web/Session/SNewUser.pmlib/ACIS/Web/Session/SOldUser.pmlib/ACIS/Web/Site.pm{HOME}/site directory as static content HTML
pages. Uses
home/presentation/default/misc/local-document.xsl.
Invoked by ACIS::Web.lib/ACIS/Web/SysProfile.pmacis.sysprof table; saves and loads
parameter-value pairs for a particular user or record. This
is used by research profile and ARPU. Generally useful
when ACIS needs to remember something about a user or a
record, without writing it to his/her userdata.
lib/ACIS/Web/User.pmlib/ACIS/Web/UserData.pmlib/ACIS/Web/Citations.pmlib/ACIS/Resources/Search.pmlib/ACIS/Resources/AutoSearch.pmlib/ACIS/Resources/Suggestions.pmrp_suggestions table.lib/ACIS/Resources/SearchFuzzy.pmlib/ACIS/APU.pmlib/ACIS/APU/Queue.pmlib/ACIS/APU/RP.pmlib/ACIS/Citations/Input.pmcitations table (and citations\_deleted)lib/ACIS/Citations/Profile.pmlib/ACIS/Citations/Search.pmlib/ACIS/Citations/SimMatrix.pmlib/ACIS/Citations/CitDocSim.pmlib/ACIS/Citations/Utils.pmlib/ACIS/Citations/Suggestions.pmcit\_sug and cit\_old\_sug tableslib/ACIS/Citations/AutoUpdate.pmlib/ACIS/Citations/Events.pmcitations\_events table maintenancelib/ACIS/Misc.pmlib/ACIS/ShortIDs.pmlib/ACIS/UserData/Data/Record.pmlib/ACIS/UserData/User/Record.pmlib/Web/App.pmWeb::App — the core of the web applications
framework. Designed to be generic, although some
project-specific stuff is still there.lib/Web/App/Common.pmlib/Web/App/XSLT.pmlib/Web/App/FormsProcessing.pmlib/Web/App/Config.pmlib/Web/App/Config/Parse.pmscreens.xml and site-local configuration
(e.g. acis.conf). This is used to make
config.bin file during ACIS setup. That file is then
reused every time ACIS::Web object is created. The module is
loaded by parse_config() method of
Web::App.lib/Web/App/Email.pmWeb::App::Email, provides email-sending services to
other modules. lib/Web/App/EmailFormat.pmlib/Web/App/Screen.pmlib/Web/App/Session.pmWeb::App::Session — session class.ACIS::Data::DumpXML is an XML-serializer, used by ACIS::Web (and Web::App). ACIS::Data::DumpXML::Parser is the de-serializer. Technically they are in ACIS:: hierarchy, but logically they belong to Web::App::.
lib/ACIS/Data/DumpXML.pmACIS::Data::DumpXML module. Converts a perl data structure into XML.
lib/ACIS/Data/DumpXML/Parser.pmAbstract metadata database system. Analyses independent metadata records and stores them in a database according to a configuration. Extracts and stores relations between records. Can retrieve records back, if necessary.
lib/ARDB.pmARDB modulelib/ARDB/Configuration.pmconfiguration.xml
lib/ARDB/ObjectDB.pmobjects table, for later
use.lib/ARDB/Table.pmARDB::Table class; represents a database table
and provides table-related methods, like create, delete,
store a record, et cetera.lib/ARDB/Common.pmlib/ARDB/Plugins.pmlib/ARDB/Record.pmARDB::Record, abstract class for metadata
objects, that ARDB can store and process.lib/ARDB/Record/ReDIF.pmlib/ARDB/Record/Simple.pmlib/ARDB/RelationType.pmlib/ARDB/Relations.pmARDB::Relations class. Manages relations
table; stores and retrieves relations between records
(objects).lib/ARDB/Relations/Transaction.pmlib/ARDB/Setup.pmARDB::Local perl-module file to
avoid re-parsing and re-checking it every time. lib/ARDB/Plugin/Processing.pmlib/ARDB/Plugin/Processing/ACIS_UD.pmlib/ARDB/Plugin/Processing/HoPEc.pmlib/ARDB/Plugin/Processing/ShortIDs.pmlib/ARDB/SiteConfig.pmlib/ARDB/Test.pmt/.lib/ARDB/ReDIF/Processing.pmlib/ARDB/AMF/Processing.pmlib/ARDB/RI.pmSQL helper module, used throughout ACIS. Provides a conveniently-wrapped interface to MySQL with problem logging.
sql_result is the class for SQL query results.
sql_helper/MANIFESTsql_helper/MANIFEST.SKIPsql_helper/Makefile.PLsql_helper/sql_helper.pmsql_helper/sql_result.pmlib/RePEc/Index/Collection/ACIS_UD.pmACIS_UD” collection type
for RePEc-Index (update daemon).
ACIS_UD is ACIS userdata files collection.lib/RePEc/Index/Collection/CitationsAMF.pmCitationsAMF” collection type
for RePEc-Index.lib/RePEc/Index/Collection/FullTextUrlsAMF.pmFullTextUrlsAMF”
collection type for RePEc-Index. This is for the FullText URLs input data.Generated: Wed Aug 29 22:59:09 2007
ACIS project, acis@openlib.org