First Workshop on
ROC Analysis in AI
Valencia, Spain, 22
August, 2004

hold within ECAI’2004, the European Conference on Artificial Intelligence.

Followed by ROCML'05 and ROCML'06.

                               Schedule                                                                                               List of accepted papers                                                  

Brief description

Receiver Operating Characteristic Analysis (ROC Analysis) is related in a direct and natural way to cost/benefit analysis of diagnostic decision making. Widely used in medicine for many decades, it has been introduced relatively recently in some areas of artificial intelligence: machine learning, multiagent systems, intelligent decision support and expert systems. In this context, ROC analysis provides tools to select possibly optimal models and to discard suboptimal ones independently from (and prior to specifying) the cost context or the class distribution. Furthermore, the Area Under the ROC Curve (AUC) has been shown to be a better evaluation measure than accuracy in contexts with variable misclassification costs and/or imbalanced datasets. AUC is also the standard measure when using classifiers to rank examples, and, hence, is used in applications where ranking is crucial, such as campaign design, model combination, collaboration strategies, and co-learning.

Nevertheless, there are many open questions and some limitations that hamper a broader use and applicability of ROC analysis. Its connections with other evaluation measures is not yet clarified completely, its incorporation in decision support and expert systems technology just envisaged, its use for improving the decisions of (communities of) intelligent agents unexplored, and its use in data mining still below its full potential. Among the limitations of ROC analysis, an important one, despite some recent progress, is its possible but difficult extension for more than two classes. 

One of the main goals of the workshop is to foster the cross-fertilisation of ideas and applications with related areas in artificial intelligence. Consequently, the presentations and discussions will be open to a broad list of topics (not exhaustive):

Details of the workshop

This is a one-day workshop (see schedule), with sessions consisting of short paper presentations, devoting an important share of time to informal discussion and interaction between the participants. The workshop will be closed with an open discussion about more promising open problems and research areas of ROC analysis, continuation of the workshop, future related events, etc. 

Papers will be selected by the program committee according to the quality of the submission and its relevance to the workshop topics. All accepted papers will be gathered in printed form and distributed to registered attendees as workshop notes.

The publication of a selected set of papers for a special volume or a journal issue is considered, but this will depend on the success and overall results of the workshop.

Accepted Papers 

List of accepted papers


Preliminary schedule

Important Dates

Workshop Program Committee

Workshop Organizing Committee

Submission Guidelines

Potential participants are invited to submit short papers (no longer than 10 pages), which may be in the form of a technical paper, a position paper (e.g. highlighting open problems or new applications of ROC analysis), an overview of their research or a software demonstration.

Authors should submit their papers electronically (PDF or PS format) to the contact person ( It is recommended to submit papers using the final camera-ready ECAI 2004 conference paper style, including author names.

Those wishing to attend the meeting without submitting a paper should send a short statement of interest to the contact person describing their work or interest in the area.

Some relevants links (introduction to ROC analysis, software, etc.)

Supported by:

Last updated: 13-Jul-2004