Publications
From TeacherWiki
Contents |
Getting Started Reading About ASSISTments
Professor Heffernan has over 30 publications related to ASSISTment, but most of them are of the types that don't interest teachers. These papers are important scientific findings that hundreds of other researchers have cited. These studies are also important as without these scientific studies, we would have no federal funding. We get funded due to our reputation as scientists producing valuable finding. So to help practitioners I have assembled a few good starting points here.
Here is a descriptive article describing how one teacher uses ASSISTment. article
Here is an article on our vision how ASSISTment could be used to kick start a Professional Learning Community Click Here article
Here is an article written for principals and superintendents.
- Militello, M., & Heffernan, N. (2009). Which one is "just right"? What educators should know about formative assessment systems. International Journal of Educational Leadership Preparation, 4(3), 1-8. [1]
Here is an article that talks about the technology behind ASSISTments.
- Razzaq, L., Patvarczki, J., Almeida, S.F., Vartak, M., Feng, M., Heffernan, N.T. and Koedinger, K. (2009). The ASSISTment builder: Supporting the Life-cycle of ITS Content Creation. IEEE Transactions on Learning Technologies Special Issue on Real-World Applications of Intelligent Tutoring Systems. 2(2) 157-166 [2]
If you are looking for documentation on how to use ASSISTments you can read this teacher manual Download the user manual PDF here and visit Getting_Started_with_ASSISTment
Research Articles with Empirical Results
We do research into assessment of student and on the best way to improve student learning. We have over two dozen such articles, but the following are two good ones to start with.
Here is an article that is focused on the assessment results.
- Feng, M., Heffernan, N.T., & Koedinger, K.R. (2009). Addressing the assessment challenge in an Intelligent Tutoring System that tutors as it assesses. The Journal of User Modeling and User-Adapted Interaction. Vol 19: p243-266. [3]
Here is a representative article concerning student learning.
- Razzaq, L. & Heffernan, N. (2009) To Tutor or Not to Tutor: That is the Question. In Dimitrova, Mizoguchi, du Boulay & Graesser (Eds.) Proceedings of the 2009 Artificial Intelligence in Education Conference. IOS Press. pp. 457-464.Honorable Mention for Best Student Paper [4]
(Here is a text version of this page .)
Empirical work Listed by Year
Everyone on of these studies listed below were peer reviewed. Accerpate rates at most of these venues are the in middle thirtys; 70% are rejected.
Recent/Accepted
Jozsef Patvarczki, Elke A. Rundensteiner, and Neil T. Heffernan, "Using Machine Learning Technique to Parallelize Databases: Where each query answered by a single node", 2010 New England DB Summit, MIT, Cambridge, 2010
Pardos, Z. A., Heffernan, N. T. (2009). Determining the Significance of Item Order In Randomized Problem Sets. In Proceedings of the 2nd International Conference on Educational Data Mining. Cordoba, Spain.
Pardos, Z. A., Heffernan, N. T. (2009) Detecting the Learning Value of Items In a Randomized Problem Set. In Proceedings of the 14th International Conference on Artificial Intelligence in Education. Brighton, UK. IOS Press.
Patvarczki, J., Politz, J., Heffernan, N.T. Scalability and Robustness in the Domain of Web Based Tutoring. In Proceedings of the 14th International Conference on Artificial Intelligence in Education, Workshop Proceedings Volume 4, Scalability Issues in AIED. Brighton, UK. 2009, pp. 20-29.
Jozsef Patvarczki, Murali Mani, and Neil Heffernan, "Performance Driven Database Design for Scalable Web Applications", Advances in Databases and Information Systems, In J. Grundspenkis, T. Morzy & G. Vossen (Eds) Advances in Databases and Information Systems Springer-Verlag: Berlin. 2009, pp 43-58
Razzaq, L., Heffernan, N.T. (To appear) To Tutor or Not to Tutor: That is the Question. To appear in the Proceedings of the Annual Conference on Artificial Intelligence in Education 2009.
Razzaq, L., Parvarczki, J., Almeida, S.F., Vartak, M., Feng, M., Heffernan, N.T. and Koedinger, K., "The ASSISTment builder: Supporting the Life-cycle of ITS Content Creation", IEEE Transactions on Learning Technologies Special Issue on Real-World Applications of Intelligent Tutoring Systems. 2(2) 2009, 157-166.
Shrestha, P., Wei, X., Maharjan, A., Razzaq, L., Heffernan, N.T., Heffernan, C. (Accepted) Are Worked Examples an Effective Feedback Mechanism During Problem Solving? Accepted to the Annual Meeting of the Cognitive Science Society, 2009.
2009
Feng, M., Heffernan, N.T., & Koedinger, K.R. (2009). Addressing the assessment challenge in an Online System that tutors as it assesses. To appear in User Modeling and User-Adapted Interaction: The Journal of Personalization Research (UMUAI journal).
Feng, M, Heffernan, N., Heffernan, C. & Mani, M. (2009). Using mixed-effects modeling to analyze different grain-sized skill models. Accepted by the IEEE Transactions on Learning Technologies Special Issue on Real-World Applications of Intelligent Tutoring Systems.
Mendicino, M., Razzaq, L. & Heffernan, N. T. (2009) Comparison of Traditional Homework with Computer Supported Homework: Improving Learning from Homework Using Intelligent Tutoring Systems. Journal of Research on Technology in Education (JRTE). Published by the International Society For Technology in Education (ISTE).
Patvarczki, J., Mani, M., and Heffernan, Neil. (2009) "Performance Driven Database Design for Scalable Web Applications", 2009 New England Database Day, MIT, Cambridge, 2009
2008
Almeida, S., Vartak, M., Heffernan, N. (2008) "Supporting the Generation of Isomorphic Tutoring Content," Proceedings of the Young Researchers Track ITS 2008, June 2008, Montreal
Ayers, E. and Junker, B.W. (2008). IRT Modeling of Tutor Performance To Predict End-of-year Exam Scores. Educational and Psychological Measurement, 68,972-987.
Guo Y., Beck, J.E., Heffernan, N.T. (2008) "Trying to Reduce Bottom-out hinting: Will telling student how many hints they have left help?", Intelligent Tutoring Systems 2008, Montreal, Canada
Feng, M., Beck, J,. Heffernan, N. & Koedinger, K. (2008) Can an Intelligent Tutoring System Predict Math Proficiency as Well as a Standardized Test? In Baker & Beck (Eds.). Proceedings of the 1st International Conference on Education Data Mining. pp.107-116. Montral 2008.
Feng, M., Heffernan, N., Beck, J, & Koedinger, K. (2008) Can we predict which groups of questions students will learn from? In Baker & Beck (Eds.). Proceedings of the 1st International Conference on Education Data Mining. pp.218-225. Montral 2008
Pardos, Z. A., Heffernan, N. T., Ruiz, C. & Beck, J. (2008). Effective Skill Assessment Using Expectation Maximization in a Multi Network Temporal Bayesian Network. The Young Researchers Track at the 20th International Conference on Intelligent Tutoring Systems. Montreal, Canada. pp. 31-40
Pardos, Z. A., Beck, J., Ruiz, C. & Heffernan, N. T. (2008). The Composition Effect: Conjunctive or Compensatory? An Analysis of Multi-Skill Math Questions in ITS. Proceedings of the First International Conference on Educational Data Mining. Montreal, Canada. pp. 147-156
Patvarczki, J., Almeida, S.F., Beck, J.E., Heffernan, N.T. (2008) "Lessons Learned from Scaling Up a Web-Based Intelligent Tutoring System", Intelligent Tutoring Systems 2008, Montreal, Canada, pp. 766-770
Razzaq, L., Mendicino, M. & Heffernan, N. (2008) Comparing classroom problem-solving with no feedback to web-based homework assistance. In Woolf, Aimeur, Nkambou and Lajoie (Eds.) Proceeding of the 9th International Conference on Intelligent Tutoring Systems. pp. 426 - 437. Springer-Verlag: Berlin.
Razzaq, L., Heffernan, N.T. (2008) Towards Designing a User-Adaptive Web-Based E-Learning System. In Mary Czerwinski, Arnold M. Lund, Desney S. Tan (Eds.): Extended Abstracts Proceedings of the 2008 Conference on Human Factors in Computing Systems, CHI 2008, pp. 3525-3530. Florence, Italy: ACM.
Razzaq, L., Patvarczki, J., Almeida, S., Vartak, M., Feng, M., Heffernan, N.T. & Koedinger, K.R. (2008) "The ASSISTment Builder: Supporting the Life Cycle of ITS Content Creation", WPI Tech report, WPI-CS-TR-08-06.
Vartak, M., Almeida, S., Heffernan, N. (2008) "Extending ITS Authoring Tools to be Culturally Aware,"Proceedings of the Workshop on Culturally Aware Tutoring Systems, June 2008, Montreal
Vartak, M., Almeida, S., Heffernan, N. (2008) "Cost-effective Content Creation with Variabilization," Technical Report Series, Computer Science Dept., W.P.I.
2007
Feng, M., Heffernan, N. T. (2007). Assessing Students. Performance Longitudinally: Item Difficulty Parameter vs. Skill Learning Tracking. Paper presented at the 2007 Annual meeting of National Council of Measurement on Educational (NCME'2007), Chicago.
Feng, M., Heffernan, N.T. (2007). Towards Live Informing and Automatic Analyzing of Student Learning: Reporting in the Assistment System. Special Issue of Journal of Interactive Learning Research (JILR) entitled "Usage Analysis in Learning Systems: Existing Approaches and Scientific Issues", 18(2).
Junker B.(2007). On-demand learning-embedded benchmark assessment using classroom-accessible technology. Proposal for a symposium at NCME Annual Conference 2007.
Pardos, Z., Feng, M. & Heffernan, N. T. & Heffernan-Lindquist, C. (2007) Analyzing fine-grained skill models using bayesian and mixed effect methods. In Luckin & Koedinger (Eds.) Proceedings of the 13th Conference on Artificial Intelligence in Education. IOS Press.
Pardos, Z. A., Heffernan, N. T., Anderson, B. & Heffernan, C. (2007). The effect of model granularity on student performance prediction using Bayesian networks. The International User Modeling Conference 2007.
Razzaq, Feng, Heffernan, Koedinger, Nuzzo-Jones, Junker, Macasek, Rasmussen, Turner & Walonoski (2007). Blending Assessment and Instructional Assistance. In Nadia Nedjah, Luiza deMacedo Mourelle, Mario Neto Borges and Nival Nunesde Almeida (Eds). Intelligent Educational Machines within the Intelligent Systems Engineering Book Series . pp.23-49. (see http://www.isebis.eng.uerj.br/). Springer Berlin / Heidelberg.
Razzaq, L., Heffernan, N., Feng, M., Pardos Z. (2007). Developing Fine-Grained Transfer Models in the ASSISTment System. Journal of Technology, Instruction, Cognition, and Learning, Vol. 5. Number 3. Old City Publishing, Philadelphia, PA. pp. 289-304.
Razzaq, L., Heffernan, N. T., Lindeman, R. W. (2007) What level of tutor interaction is best? In Luckin & Koedinger (Eds) Proceedings of the 13th Conference on Artificial Intelligence in Education. IOS Press. pp. 222-229.
Wietz, R., Heffernan, N. T., Kodaganallur, V. & Rosenthal, D. (2007). The distribution of student errors across schools: An initial study. In Luckin & Koedinger (Eds.) Proceedings of the 13th Conference on Artificial Intelligence in Education. IOS Press.
2006
Anozie N., & Junker B. W. (2006). Predicting end-of-year accountability assessment scores from monthly student records in an online tutoring system. In Beck, J., Aimeur, E., & Barnes, T. (Eds). Educational Data Mining: Papers from the AAAI Workshop. Menlo Park, CA: AAAI Press. pp. 1-6. Technical Report WS-06-05.
Ayers E., & Junker B. W. (2006). Do skills combine additively to predict task difficulty in eighth grade mathematics? In Beck, J., Aimeur, E., & Barnes, T. (Eds). Educational Data Mining: Papers from the AAAI Workshop. Menlo Park, CA: AAAI Press. pp. 14-20. Technical Report WS-06-05.
Feng, M., Heffernan, N.T., & Koedinger, K.R. (2006a). Addressing the Testing Challenge with a Web-Based E-Assessment System that Tutors as it Assesses. Proceedings of the Fifteenth International World Wide Web Conference. pp. 307-316. New York, NY: ACM Press. 2006. ISBN:1-59593-332-9. Nominated for "Best Student Paper".
Feng, M., Heffernan, N.T., & Koedinger, K.R. (2006b). Predicting state test scores better with intelligent tutoring systems: developing metrics to measure assistance required. In Ikeda, Ashley & Chan (Eds.). Proceedings of the 8th International Conference on Intelligent Tutoring Systems. Springer-Verlag: Berlin. pp. 31-40. 2006.
Feng, M., Heffernan, N. T., Mani, M., & Heffernan, C. (2006). Using Mixed-Effects Modeling to Compare Different Grain-Sized Skill Models. In Beck, J., Aimeur, E., & Barnes, T. (Eds). Educational Data Mining: Papers from the AAAI Workshop. Menlo Park, CA: AAAI Press. pp. 57-66. Technical Report WS-06-05. ISBN 978-1-57735-287-7.
Feng, M., Heffernan, N.T. (2006). Informing Teachers Live about Student Learning: Reporting in the Assistment System. To be published in Technology, Instruction, Cognition, and Learning Journal Vol. 3. Old City Publishing, Philadelphia, PA. 2006.
Heffernan N.T., Turner T.E., Lourenco A.L.N., Macasek M.A., Nuzzo-Jones G., Koedinger K.R., The ASSISTment Builder: Towards an Analysis of Cost Effectiveness of ITS creation, Accepted by FLAIRS2006, Florida, USA (2006).
Junker, B. W. (2006). Using On-line Tutoring Records to Predict End-of-Year Exam Scores: Experience with the ASSISTments Project and MCAS 8th Grade Mathematics. To appear in Lissitz, R. W. (Ed.), Assessing and modeling cognitive development in school: intellectual growth and standard settings. Maple Grove, MN: JAM Press.
Kardian, K., Heffernan, N.T. (2006). Knowledge Engineering for Intelligent Tutoring Systems: Assessing Semi-automatic Skill Encoding Methods. In Ikeda, Ashley & Chan (Eds.). Proceedings of the 8th International Conference on Intelligent Tutoring Systems. Springer-Verlag: Berlin. pp. 735-737
Pardos, Z. A., Heffernan, N. T., Anderson, B., & Heffernan C. (2006). Using Fine-Grained Skill Models to Fit Student Performance with Bayesian Networks. Workshop in Educational Data Mining held at the 8th International Conference on Intelligent Tutoring Systems. Taiwan. 2006.
Rasmussen, K., Patvarczki J. & Heffernan, N.T. (2006) "Scaling up the Server-based ASSISTment ITS: Speed and Reliability", WPI Tech Report, WPI-CS-TR-06-12.
Razzaq, L., Heffernan, N.T. (2006). Scaffolding vs. hints in the Assistment System. In Ikeda, Ashley & Chan (Eds.). Proceedings of the 8th International Conference on Intelligent Tutoring Systems. Springer-Verlag: Berlin. pp. 635-644. 2006.
Walonoski, J., Heffernan, N.T. (2006). Detection and Analysis of Off-Task Gaming Behavior in Intelligent Tutoring Systems. In Ikeda, Ashley & Chan (Eds.). Proceedings of the 8th International Conference on Intelligent Tutoring Systems. Springer-Verlag: Berlin. pp. 382-391. 2006
Walonoski, J., Heffernan, N. T. (2006). Prevention of Off-Task Gaming Behavior in Intelligent Tutoring Systems. In Ikeda, Ashley & Chan (Eds.). Proceedings of the 8th International Conference on Intelligent Tutoring Systems. Springer-Verlag: Berlin. pp. 722-724. 2006
2005
Feng, M., Heffernan, N.T. (2005). Informing Teachers Live about Student Learning: Reporting in the Assistment System. the 12th International Conference on Artificial Intelligence in Education 2005 Workshop on Usage Analysis in Learning Systems, 2005, Amsterdam.
Feng, M., Heffernan, N.T., & Koedinger, K.R. (2005). Looking for Sources of Error in Predicting Student's Knowledge. In Beck. J (Eds). Educational Data Mining: Papers from the 2005 AAAI Workshop. Menlo Park, California: AAAI Press. pp. 54-61 Technical Report WS-05-02. ISBN 978-1-57735-238-9
Nuzzo-Jones, G., Walonoski, J.A., Heffernan, N.T., Livak, T. (2005). The eXtensible Tutor Architecture: A New Foundation for ITS. In C.K. Looi, G. McCalla, B. Bredeweg, & J. Breuker (Eds.) Proceedings of the 12th International Conference on Artificial Intelligence In Education, 902-904. Amsterdam: ISO Press.
Nuzzo-Jones, G., Walonoski, J.A., Heffernan, N.T., Livak, T. (2005). The eXtensible Tutor Architecture: A New Foundation for ITS. In Proceedings of the 12th International Conference on Artificial Intelligence in Education 2005 Workshop on Adaptive Systems for Web-Based Education: Tools and Reusability, Amsterdam
Razzaq, L., Feng, M., Nuzzo-Jones, G., Heffernan, N.T., Koedinger, K. R., Junker, B., Ritter, S., Knight, A., Aniszczyk, C., Choksey, S., Livak, T., Mercado, E., Turner, T.E., Upalekar. R, Walonoski, J.A., Macasek. M.A., Rasmussen, K.P. (2005). The Assistment Project: Blending Assessment and Assisting. In C.K. Looi, G. McCalla, B. Bredeweg, & J. Breuker (Eds.) Proceedings of the 12th International Conference on Artificial Intelligence In Education, 555-562. Amsterdam: ISO Press.
Rose, C., Donmez, P., Gweon, G., Knight, A., Junker, B., Cohen, W., Koedinger, K., Heffernan, N. (2005). Automatic and Semi-Automatic Skill Coding With a View Towards Supporting On-Line Assessment. the 12th International Conference on Artificial Intelligence in Education 2005, Amsterdam
Turner, T.E., Macasek, M.A., Nuzzo-Jones, G., Heffernan, N..T, Koedinger, K. (2005). The Assistment Builder: A Rapid Development Tool for ITS. In C.K. Looi, G. McCalla, B. Bredeweg, & J. Breuker (Eds.) Proceedings of the 12th International Conference on Artificial Intelligence In Education, 929-931. Amsterdam: ISO Press.
Turner, T.E., Macasek, M.A., Nuzzo-Jones, G., Heffernan, N..T, Koedinger, K. (2005). The Assistment Builder: A Rapid Development Tool for ITS. Poster in the 12th International Conference on Artificial Intelligence in Education 2005 Workshop on Adaptive Systems for Web-Based Education: Tools and Reusability, Amsterdam
