Ilya M. Goldin

I've recently defended my dissertation, titled "A Focus on Content: The Use of Rubrics in Peer Review to Guide Students and Instructors." [PDF]  [Abstract]

Together with Christian Schunn and Kevin Ashley, we are co-editing a special issue of the Journal of Writing Research: Redesigning Peer Review Interactions Using Computer Tools. Submissions have now closed.

The proceedings of Computer-Supported Peer Review in Education Workshop (CSPRED) are now available. CSPRED, a workshop I co-chaired, was held in conjunction with the Tenth International Conference on Intelligent Tutoring Systems (ITS 2010).

At present, I am a Consulting Principal Investigator and Research Scientist for several academic and industry projects, including:

• mining relationships among patients and medical care providers to help prevent patients' disengagement from care
• improving predictive assessment of professional skills in law
• helping experts on chronic diseases educate patients about disease self-management

I've recently defended my PhD dissertation in the Intelligent Systems Program at Pitt, titled "A Focus on Content: The Use of Rubrics in Peer Review to Guide Students and Instructors." [PDF]  [Abstract] The research was performed under the direction of Kevin Ashley, who is a Senior Scientist at the Learning Research & Development Center. The dissertation examined how students can give formative feedback to each other in peer review, and how hierarchical Bayesian models can be used to mine that feedback to inform an instructor or an intelligent tutoring system. To conduct this research, I built Comrade, a web-based application for peer review. This research was supported in part by a grant from the Provost's Advisory Council on Instructional Excellence, "A Peer-Review-Based Student Model for Ill-Defined Problem-solving", Kevin Ashley, PI.

Visit me on LinkedIn.

Say 'hi' to my Mom! She is a piano teacher in Germantown, MD.

Some photos from the ITS 2010 conference, including the Phipps Conservatory banquet, are now available.

From 2006-2009, I worked as a researcher at JustSystems Evans Research (formerly Clairvoyance Corp.), a CMU spin-off founded by David A. Evans. JSERI closed in December of 2009. This work included data mining and visualization, recommender systems, information retrieval and extraction, entity recognition, and many other AI problems.

For my Masters thesis, I collaborated with Kevin Ashley and Rosa Pinkus on an assessment of an evaluation tool for case analysis in bioengineering ethics. Engineers, and bioengineers in particular, must know how to address ethical problems that arise in their work, and must therefore be trained in ethics case analysis. Assessment of learning outcomes in this context must be flexible enough to allow for the wide variety of ways in which students can frame the cases they analyze. I led the evaluation of an assessment tool that addresses this need, partially supported by the National Science Foundation (award #0203307).

Before that, I worked on PETE, the Professional Ethics Tutoring Environment. PETE is a web application aimed at helping bioengineering students practice ethics case analysis. When it was developed, PETE was unique among software for ethics pedagogy thanks to its combination of features:

• availability over the Web
• focus on a formal method of ethics case analysis
• allowing students to compare their work to the work of their peers

Due to technical reasons, PETE is no longer available online. NSF also funded this research (award #9720341).

conf Ashley, Kevin D., Ilya M. Goldin. (in preparation) "Toward AI-enhanced Computer-supported Peer Review in Legal Education."

journal Goldin, Ilya M., Kevin D. Ashley. (under review) "Conceptually focusing peer feedback using rating dimensions."

conf Goldin, Ilya M., Kevin D. Ashley. (2011) "Peering Inside Peer Review with Bayesian Models." Proceedings of the 15th International Conference on Artificial Intelligence in Education. New Zealand [PDF]

  Goldin, Ilya M., Kevin D. Ashley. (2010) "Learning by Reviewing through Peer Feedback Refinement." Proceedings of the Workshop on Computer-Supported Peer Review in Education, 10th International Conference on Intelligent Tutoring Systems. Pittsburgh, USA [PDF]

conf Goldin, Ilya M., Kevin D. Ashley. (2010) "Eliciting Informative Feedback in Peer Review: Importance of Problem-Specific Scaffolding." Proceedings of the 10th International Conference on Intelligent Tutoring Systems. Pittsburgh, USA. [PDF]

journal Goldin, Ilya M., Rosa L. Pinkus, Kevin D. Ashley. (accepted) "Validity and Reliability of an Instrument for Assessing Case Analyses in Bioengineering Ethics Education." Science and Engineering Ethics. [Available by request before publication]

conf Goldin, Ilya M., Kevin D. Ashley, Rosa L. Pinkus. (2006) "Assessing Case Analyses in Bioengineering Ethics Education: Reliability and Training." Proceedings of the International Conference on Engineering Education. San Juan, Puerto Rico. [PDF]

  Goldin, Ilya M., Kevin D. Ashley, Rosa L. Pinkus. (2006) "Teaching Case Analysis through Framing: Prospects for an ITS in an ill-defined domain." Proceedings of the Workshop on Intelligent Tutoring Systems for Ill-Defined Domains, 8th International Conference on Intelligent Tutoring Systems, Jhongli, Taiwan. [PDF]

  Goldin, Ilya M., Wendy W. Chapman. "Learning to Detect Negation with 'Not' in Medical Texts." Proceedings of the Workshop on Text Analysis and Search for Bioinformatics at the 26th Annual International ACM SIGIR Conference (SIGIR-2003). Eds. Eric Brown, William Hersh and Alfonso Valencia. [PDF]

conf Goldin, Ilya M., Kevin D. Ashley, Rosa L. Pinkus. "Introducing PETE: Computer Support for Teaching Ethics." Proceedings of the Eighth International Conference on Artificial Intelligence & Law (ICAIL-2001). Eds. Henry Prakken and Ronald P. Loui. Association of Computing Machinery, New York. ©Copyright 2001 by ACM, Inc. [PDF]

  Evans, David A.; Bennett, Jeffrey; Hull, David; Cheng, Hua; Qu, Yan; Tenny, Carol; Montgomery, Jesse; Goldin, Ilya. "Method and Apparatus for Anchoring Expressions Based on an Ontological Model of Semantic Information." U.S. Patent Application 20080294426

  Hull, David; Evans, David A.; Bennett, Jeffrey; Cheng, Hua; Qu, Yan; Tenny, Carol; Montgomery, Jesse; Goldin, Ilya. "Method and Apparatus for Performing Semantic Update and Replace Operations." U.S. Patent Application 20080294425

  Evans, David A.; Bennett, Jeffrey; Hull, David; Cheng, Hua; Qu, Yan; Tenny, Carol; Montgomery, Jesse; Goldin, Ilya. "Method and Apparatus for Performing Semantically Informed Text Operations." U.S. Patent Application 20080295013

  Cheng, Hua; Evans, David A.; Bennett, Jeffrey; Hull, David; Qu, Yan; Tenny, Carol; Montgomery, Jesse; Goldin, Ilya. "Method and Apparatus for Performing a Semantically Informed Merge Operation." U.S. Patent Application 20080294427

  Qu, Yan; Evans, David A.; Goldin, Ilya. "Method and Apparatus for the Automated Construction of Models of Activities from Textual Descriptions of the Activities." U.S. Patent Application 20080294398

"A Peer Review-Based Student Model for Ill-Defined Problem-Solving". Provost's Advisory Council on Instructional Excellence, 2009

"Assessing Case Analyses in Bioengineering Ethics". Outstanding Paper Presentation Award, University of Pittsburgh Grad Expo 2006.

"Introducing PETE: Computer Support for Teaching Ethics". Outstanding Paper Presentation Award, University of Pittsburgh Grad Expo 2001.

Faculty of Arts and Sciences Fellowship, University of Pittsburgh, 1999-2000.

President's Award, Villa Julie College, 1999.

Outstanding Scholar Award, Villa Julie College, 1999.

First place, Computer Scholarship Competition, Villa Julie College, 1996.

Program committee member, 10th International Conference on Creating, Connecting and Collaborating through Computing

Guest co-editor, Journal of Writing Research, Special Issue: Redesigning Peer Review Interactions Using Computer Tools. With Christian Schunn and Kevin Ashley.

Reviewer, Journal of Educational Data Mining (JEDM)

Reviewer, International Conference on Artificial Intelligence in Education (AIED), 2011

Chair, Workshop on Computer-Supported Peer Review in Education (CSPRED), 10th International Conference on Intelligent Tutoring Systems, 2010

Reviewer, International Conference on Educational Data Mining (EDM), 2010

Reviewer, International Conference on Computers and Education (ICCE), 2009

Reviewer, Conference of the Florida Artificial Intelligence Research Society (FLAIRS), 2008, 2009

Grant application reviewer, Economic and Social Research Council, UK, 2008

Undergraduate Student Researcher, Center for Research in Information Access, Columbia University, summer of 1999. Natural Language Processing work with Dr Nina Wacholder and Dr Judith Klavans.

BS Magna Cum Laude, Liberal Arts and Technology, Computer Information Systems, Villa Julie College, 1996-1999.

Vice-President, Villa Julie College Chapter of the Association of Information Technology Professionals, 1998-1999 (when I founded the Chapter Scholarship).

Intern, Research and Development, McCabe & Associates, 1997-1998.