Overview
Data processing
Summary
Web interface
Summary
A brief guide to the ACIS source files
A detailed guide to the ACIS source files
Traditional meta files
Documentation
Installation and configuration
Presenters and other presentation-related files
General
Users’ screens
Person-profile editing screens
New user (initial registration) screens
Administrative screens
Email generation
Other
ACIS:: hierarchy
ACIS::Web — Web interface of ACIS
Research Profile underlying modules
APU
Citations
Other ACIS:: modules
Web::App — the web application framework
ACIS::Data::DumpXML
ARDB — the data-processing
SQL helper
Other
Internally, ACIS consists of two main subsystems and a number of smaller tools. These two main parts are very different and do not know much about each other; they work or can work pretty independently. First there is the data processing subsystem. Second there is the web interface subsystem. The glue that makes them perform together as a single application is the predefined and coordinated configuration, some shared modules and a bunch of scripts.
The data processing subsystem processes input metadata. It puts the data into database tables for the web interface to use. The web interface subsystem handles user requests arriving through the web server. Sometimes it creates data files, which data processing subsystem will process. The system is complex, but consists of simple parts; each part has distinct responsibilities.
The data processing subsystem, in its turn, consists of two loosely coupled parts. The first part monitors data files, tracks their modifications, keeps records about data which is found in those files and filters out data records, which have bad identifiers. For historical reasons, it is called RePEc-Index, RI for short.
The RePEc-Index is built around a simple idea that metadata comes from
collections. Each collection has a name (identifier), a
type and is stored in data files somewhere in the
filesystem. Metadata is only useful when someone processes it for
something. So each collection may have further processing
defined for it. All these things are specified in the collections
configuration. ACIS creates this configuration for RePEc-Index
in the file RI/collections
.
In a collection, each file may contain zero, one or many data records. Each data record must have a unique identifier. If two or more records in a collection have the same identifier, there is a conflict. RePEc-Index excludes the conflicting records from further processing, until there’s only one record left with a particular id. When a data record is successfully read from a data file, RePEc-Index does its checks and, if everything is ok, executes further processing for it.
That is when the second part of data processing comes into play. The second part is ARDB (abbreviation from Abstract RePEc DataBase, again with historical reasons). RI sends ARDB a record object, and ARDB processes it. To process a record may mean to extract certain pieces of information from it, run arbitrary perl code on it, store it in a database table. Similarly, RePEc-Index calls ARDB when a record disappears (or when RI discovers an identifier conflict). ARDB then cleans up the database. It removes the data which originated from that record.
ARDB’s work is governed by an elaborate configuration.
The configuration is stored in
(configuration.xml
in the ACIS home directory. The
configuration specifies what ARDB has to do when
processing a record of a specific type. Possible
kinds of actions: store some of the record’s data into a
database table, execute certain perl code, extract a
relationship of this record to some other record. When
cleaning up a record, ARDB can remove database table records
or call perl code.
ARDB’s configuration also defines database tables. Those may be tables, which other parts of the configuration refer to, but they don’t have to. Either way, ARDB can create those tables for you; there is a script for that (bin/create_tables). ACIS relies on this capability of ARDB to create tables which are needed.
Now, let me again outline the workflow of data processing. When RePEc-Index finds a new or changed data record, and this record has a valid unique identifier, it asks ARDB to process it. When RePEc-Index finds that a record has been removed from a data file, it asks ARDB to remove the record’s data. Also, when there is an id conflict, RePEc-Index will request ARDB to clean up the conflicting record’s data. The connection between RePEc-Index and ARDB is not intrinsic; it is just in the way of how ACIS configures RI to process the metadata.
ACIS consists of data processing part and web-interface part. There are also some other smaller tools, which serve as glue.
Data processing system is RePEc-Index and ARDB. RePEc-Index monitors data files for changes and checks records for id uniqueness.
ARDB stores data in database tables and cleans it up when necessary. The work it does is specified in an elaborate configuration.
In ACIS, RePEc-Index is configured to call ARDB for the actual data processing.
The web interface part of ACIS is:
Perl code: CGI script + framework modules + ACIS-specific modules + helper scripts
a set of XSLT files called presenters
web application configuration, screens.xml
There are two main parts in the set of perl modules that
make ACIS web interface work. First is the framework. It
contains general code for analysing a user’s request,
deciding how to treat that request and generating a
response. It helps ACIS to store and load user sessions and
does a bunch of other little things for a web application.
That’s why I call it the web application framework. It’s
core is in the Web::App
module.
The second part contains the specifics of ACIS. There lives
code about ACIS user accounts, searching for research items
and research institutions, creating and updating a personal
profile and so on. This part is rooted at the
ACIS::Web
module. The framework (ie. the previous
part) makes extending and debugging this part much easier by
providing tools and a common environment.
These two parts are tied together by:
inheritance — ACIS::Web
class inherits from
Web::App
and extends it in ACIS-specific ways.
screens.xml
— the application configuration.
The application configuration, the screens.xml
file,
is built around the notion of a screen. Screens are basic
units of the web interface; they handle incoming requests
and generate responses. Each screen represents a certain
piece of web-accessible functionality of the system.
Web::App looks at addresses (URLs) of the requests that
arrive. For each request it decides which screen it is for.
Each screen definition in screens.xml
configuration
specifies which modules and which functions will
Web::App invoke for it. If a request came for an
unknown (unspecified) screen, Web::App will generate
a 404 error and display a “Sorry” screen.
Another thing that the screens configuration brings into the mix is presenters. For each screen it defines an XSLT file, which is used to generate a response page. This means the application presentation is strictly separate from the main application logic.
All this separation between web-application general and ACIS-specific, between presentation and application logic works for flexibility and extensibility of the system. It is an attempt to follow the “separation of concerns” principle.
At the same time, the CGI script in ACIS is so simple that I
could have left it unmentioned without harm. Basically it
creates an object and calls a method on it (or two) and there it
ends. The other web-interface-related scripts mostly act in
a similar way: they involve a certain part of the
ACIS::Web
and it does the rest.
ACIS is built on top of a generic web-application framework.
ACIS::Web module and many other modules in
the ACIS::*
hierarchy encapsulate all what’s
essentially ACIS-specific.
All of the user interface is coded in XSLT stylesheets (templates).
Application configuration (screens.xml
) relates
different pieces together, uniting all of the above into a
single solid application.
Main parts and some key files
This is an annotated list of most files of an ACIS distribution.
COPYING
MANIFEST
MANIFEST.SKIP
Makefile.PL
README
TODO
doc/make.linked.pl
xsltproc
in PATH.
doc/make.simple.pl
doc/check-filelist.pl
doc/internal.text
and in MANIFEST. Obsolete.
doc/style.css
doc/xslt/010.xsl
doc/xslt/020.xsl
doc/xslt/030.xsl
doc/index.html
doc/adm.html
doc/apache-conf.html
doc/conf.html
doc/install.html
doc/bdb-private.html
doc/overview.html
doc/researchprofile.html
doc/apu.html
doc/cooperate.html
doc/eprints.html
doc/eprints-install.html
doc/overview.html
doc/citations.html
doc/db.html
doc/daemon.html
doc/internal.html
doc/*.text
home/bin/conf.pl
main.conf
and creates thisconf.sh
,
ardb.conf
, acis.conf
home/bin/rid
home/bin/setup
home/bin/setup.cgi_frontend
home/bin/setup.logs-browsing
home/bin/setup.ri_collections
home/bin/setup.ri_local_setup.pm
home/bin/setup.sid_local.pm
home/bin/templates/apu.pl
home/bin/templates/clean-up.pl
home/bin/templates/create_tables.pl
home/bin/templates/upgrade_to_*.pl
home/bin/templates/*.pl
home/configuration.xml
home/contributions.conf.xml
main.conf.eg
home/screens.xml
install.sh
home/presentation/default/global.xsl
show-status
template. home/presentation/default/page.xsl
page
template) and a special markup, used all over
the place. Directly or indirectly it is used by every
HTML-page presenter of ACIS.home/presentation/default/forms.xsl
home/presentation/default/page-universal.xsl
appropriate-page
and
appropriate-page-soft
template, which display
new-user-page
to new users and user-page
to returning users.home/presentation/default/errors.xml
error()
method in
Web::App. home/presentation/default/messages.xml
message()
method in Web::App. home/presentation/default/fields.xml
show-status
template.home/presentation/default/fields-institution.xml
fields.xml
(see previous item)
in case of the new-institution
screen.home/presentation/default/index.xsl
home/presentation/default/misc/login-pass.xsl
home/presentation/default/misc/login.xsl
home/presentation/default/misc/sorry.xsl
home/presentation/default/misc/local-document.xsl
ACIS::Web::Site
.home/presentation/default/phrase.xml
<phrase ref=’…’/>
markup elsewhere in
templates. Such a phrase element will be replaced with
content from this file or its installation-local equivalent
{HOME}/presentation/default/phrase-local.xml
. home/presentation/default/script/main.js
home/presentation/default/script/jquery.js
home/presentation/default/style/brownish.css.add
home/presentation/default/style/ie-font-sizes.css
home/presentation/default/style/main.css
home/presentation/default/user/page.xsl
user-page
template for all users’ screens.home/presentation/default/user/welcome.xsl
home/presentation/default/user/settings.xsl
settings
screen.home/presentation/default/user/good-bye.xsl
home/presentation/default/user/unregister.xsl
home/presentation/default/user/account-deleted.xsl
home/presentation/default/person/page.xsl
home/presentation/default/person/affiliations-common.xsl
home/presentation/default/person/affiliations-ir-guide.xsl
home/presentation/default/person/affiliations-search.xsl
home/presentation/default/person/affiliations.xsl
home/presentation/default/person/affiliations/new-institution.xsl
home/presentation/default/person/contact.xsl
home/presentation/default/person/name.xsl
home/presentation/default/person/interests.xsl
home/presentation/default/person/photo.xsl
home/presentation/default/person/profile-overview.xsl
home/presentation/default/person/profile-static.xsl
home/presentation/default/person/profile-show.xsl
profile-overview.xsl
and profile-static.xsl
home/presentation/default/person/research/*.xsl
home/presentation/default/citations/*.xsl
home/presentation/default/person/generic.xsl
home/presentation/default/new-user/page.xsl
new-user-page
template.home/presentation/default/new-user/initial.xsl
home/presentation/default/new-user/additional.xsl
home/presentation/default/new-user/complete.xsl
home/presentation/default/new-user/confirm.xsl
home/presentation/default/adm/events-decode.xsl
/adm/events/decode
screen
home/presentation/default/adm/events-raw.xsl
/adm/events/raw
screen
home/presentation/default/adm/events.xsl
home/presentation/default/adm/index.xsl
/adm
screen
home/presentation/default/adm/pass.xsl
/adm/…
screens
home/presentation/default/adm/search-res-doc.xsl
home/presentation/default/adm/search-res-rec.xsl
home/presentation/default/adm/search-res-usr.xsl
home/presentation/default/adm/search.xsl
home/presentation/default/adm/session-deleted.xsl
home/presentation/default/adm/session.xsl
home/presentation/default/adm/sessions.xsl
home/presentation/default/adm/sql.xsl
home/presentation/default/email/general.xsl
home/presentation/default/email/*.xsl
home/presentation/default/misc/forgotten-password.xsl
home/presentation/default/widgets.xsl
home/presentation/default/indent.xsl
home/presentation/default/export/amf-person.xsl
home/presentation/default/export/redif.xsl
home/presentation/default/misc/time.xsl
home/presentation/default/misc/time-test-data.xml
home/presentation/default/stub.xsl
lib/ACIS/Web.pm
lib/ACIS/Web/Admin.pm
lib/ACIS/Web/Admin/Events.pm
/adm/events/decode
and /adm/events/raw
.
lib/ACIS/Web/Affiliations.pm
lib/ACIS/Web/Background.pm
ACIS::Web::Background
module. Its all about forking
a process and keeping track of the forked threads.
lib/ACIS/Web/CGI/Untaint/latinname.pm
lib/ACIS/Web/CGI/Untaint/name.pm
lib/ACIS/Web/CGI/Untaint/password.pm
lib/ACIS/Web/CGI/Untaint/simpleemail.pm
lib/ACIS/Web/CGI/Untaint/url.pm
lib/ACIS/Web/Config.pm
lib/ACIS/Web/Contributions.pm
lib/ACIS/Web/Export.pm
lib/ACIS/Web/Import.pm
lib/ACIS/Web/NewUser.pm
lib/ACIS/Web/Person.pm
lib/ACIS/Web/SaveProfile.pm
lib/ACIS/Web/Services.pm
lib/ACIS/Web/Session.pm
lib/ACIS/Web/Session/SMagic.pm
lib/ACIS/Web/Session/SNewUser.pm
lib/ACIS/Web/Session/SOldUser.pm
lib/ACIS/Web/Site.pm
{HOME}/site
directory as static content HTML
pages. Uses
home/presentation/default/misc/local-document.xsl.
Invoked by ACIS::Web.lib/ACIS/Web/SysProfile.pm
acis.sysprof
table; saves and loads
parameter-value pairs for a particular user or record. This
is used by research profile and ARPU. Generally useful
when ACIS needs to remember something about a user or a
record, without writing it to his/her userdata.
lib/ACIS/Web/User.pm
lib/ACIS/Web/UserData.pm
lib/ACIS/Web/Citations.pm
lib/ACIS/Resources/Search.pm
lib/ACIS/Resources/AutoSearch.pm
lib/ACIS/Resources/Suggestions.pm
rp_suggestions
table.lib/ACIS/Resources/SearchFuzzy.pm
lib/ACIS/APU.pm
lib/ACIS/APU/Queue.pm
lib/ACIS/APU/RP.pm
lib/ACIS/Citations/Input.pm
citations
table (and citations\_deleted
)lib/ACIS/Citations/Profile.pm
lib/ACIS/Citations/Search.pm
lib/ACIS/Citations/SimMatrix.pm
lib/ACIS/Citations/CitDocSim.pm
lib/ACIS/Citations/Utils.pm
lib/ACIS/Citations/Suggestions.pm
cit\_sug
and cit\_old\_sug
tableslib/ACIS/Citations/AutoUpdate.pm
lib/ACIS/Citations/Events.pm
citations\_events
table maintenancelib/ACIS/Misc.pm
lib/ACIS/ShortIDs.pm
lib/ACIS/UserData/Data/Record.pm
lib/ACIS/UserData/User/Record.pm
lib/Web/App.pm
Web::App
— the core of the web applications
framework. Designed to be generic, although some
project-specific stuff is still there.lib/Web/App/Common.pm
lib/Web/App/XSLT.pm
lib/Web/App/FormsProcessing.pm
lib/Web/App/Config.pm
lib/Web/App/Config/Parse.pm
screens.xml
and site-local configuration
(e.g. acis.conf
). This is used to make
config.bin
file during ACIS setup. That file is then
reused every time ACIS::Web object is created. The module is
loaded by parse_config()
method of
Web::App.lib/Web/App/Email.pm
Web::App::Email
, provides email-sending services to
other modules. lib/Web/App/EmailFormat.pm
lib/Web/App/Screen.pm
lib/Web/App/Session.pm
Web::App::Session
— session class.ACIS::Data::DumpXML is an XML-serializer, used by ACIS::Web (and Web::App). ACIS::Data::DumpXML::Parser is the de-serializer. Technically they are in ACIS:: hierarchy, but logically they belong to Web::App::.
lib/ACIS/Data/DumpXML.pm
ACIS::Data::DumpXML
module. Converts a perl data structure into XML.
lib/ACIS/Data/DumpXML/Parser.pm
Abstract metadata database system. Analyses independent metadata records and stores them in a database according to a configuration. Extracts and stores relations between records. Can retrieve records back, if necessary.
lib/ARDB.pm
ARDB
modulelib/ARDB/Configuration.pm
configuration.xml
lib/ARDB/ObjectDB.pm
objects
table, for later
use.lib/ARDB/Table.pm
ARDB::Table
class; represents a database table
and provides table-related methods, like create, delete,
store a record, et cetera.lib/ARDB/Common.pm
lib/ARDB/Plugins.pm
lib/ARDB/Record.pm
ARDB::Record
, abstract class for metadata
objects, that ARDB can store and process.lib/ARDB/Record/ReDIF.pm
lib/ARDB/Record/Simple.pm
lib/ARDB/RelationType.pm
lib/ARDB/Relations.pm
ARDB::Relations
class. Manages relations
table; stores and retrieves relations between records
(objects).lib/ARDB/Relations/Transaction.pm
lib/ARDB/Setup.pm
ARDB::Local
perl-module file to
avoid re-parsing and re-checking it every time. lib/ARDB/Plugin/Processing.pm
lib/ARDB/Plugin/Processing/ACIS_UD.pm
lib/ARDB/Plugin/Processing/HoPEc.pm
lib/ARDB/Plugin/Processing/ShortIDs.pm
lib/ARDB/SiteConfig.pm
lib/ARDB/Test.pm
t/
.lib/ARDB/ReDIF/Processing.pm
lib/ARDB/AMF/Processing.pm
lib/ARDB/RI.pm
SQL helper module, used throughout ACIS. Provides a conveniently-wrapped interface to MySQL with problem logging.
sql_result is the class for SQL query results.
sql_helper/MANIFEST
sql_helper/MANIFEST.SKIP
sql_helper/Makefile.PL
sql_helper/sql_helper.pm
sql_helper/sql_result.pm
lib/RePEc/Index/Collection/ACIS_UD.pm
ACIS_UD
” collection type
for RePEc-Index (update daemon).
ACIS_UD
is ACIS userdata files collection.lib/RePEc/Index/Collection/CitationsAMF.pm
CitationsAMF
” collection type
for RePEc-Index.lib/RePEc/Index/Collection/FullTextUrlsAMF.pm
FullTextUrlsAMF
”
collection type for RePEc-Index. This is for the FullText URLs input data.Generated: Wed Aug 29 22:59:09 2007
ACIS project, acis@openlib.org