Minutes: International Scientific Committee Meeting 6 Time/Date: 14:45 -- 17:45 (incl. break), 17th May 2002 Place: KAIST, Daejeon, Korea Status: Approved (September 2002, by e-mail) Present (alphabetic order): Kyung-Yong Chwa Host n Greg Galparin Host n+1 Jyrki Nummenmaa Host n-1 Djura Paunic Elected Tom Verhoeff Elected (chair) Absent (with notification): Ian Munro Elected Kunsoo Park Host n 1. The meeting was opened by Tom Verhoeff. 2. The agenda was approved. 3. The minutes of the previous meeting had already been discussed and approved by e-mail last year. 4. A considerable amount of time of the review was spent on getting to know the details of the tasks, their grading and their background. Also the grading system has been extensively discussed, more so than last year. Finally, work was done on polishing the formulation of the tasks and on producing a written report detailing the findings. Tomorrow, a draft version of the review report will be presented. The review report summarizes all findings and recommendations of the past week. It consists of three parts. The first part is very general --mostly concerning the competition site-- and it will also be made available to Ha-Jine Kimn. The second part concerns the IOI2002 competition material, but it contains no information about individual tasks. The third part contains all task-specific items. The ISC will continue to be involved in the preparation of IOI2002. In particular, Greg is still involved as task author. Jyrki, Ian, Djura and Greg will polish task descriptions by the end of May. Several members will arrive earlier at IOI2002 to assist in the final preparations. 5. This year, the actual review lasted six days, as opposed to five in the preceding two years. More time was needed because the grading system needed extra attention. All agreed that doing the review in the host country is indispensible, because o it guarantees that the ISC members will be less distracted by other duties, o it is easy to meet and consult Host SC members and assistants, o additional competition-related material can readily be made available on short notice without much security risks. A visit to the actual competition site is helpful, but not a necessity, because the IC must have inspected that in their earliervisit. The timing of the review this year --three months prior to the actual IOI-- was a difficult compromise. It was expressed that meeting with the full committee was preferred. The process of the review seemed appropriate. 6. We discussed the upcoming IOI2002 competition, in particular, the role of the ISC members and the procedure for task approval in the GA meeting preceding the task translation. Kyung-Yong expressed that he would appreciate the help of ISC members in communicating to the GA. He also proposed to link each of the IOI2002 candidate tasks to a non-Host ISC member, who would be involved in polishing the final wording and in resolving any issues that arise concerning this task during IOI2002. The meeting decided to give this a try. At IOI2001, the task approval procedure had been polished, based on a proposal by the German and Polish delegations. That procedure was not strictly followed, and left some room for further improvement. Tom will write up a refinement. This procedure will be presented to the GA in the first GA meeting at IOI2002. The IC must also be informed of this intention. 7. Concerning the IOI2003 Host SC, Don Piele reported the following through e-mail. "I cannot say for sure until after I meet with Rob next week (May 22-23) in Colorado who will be the chair of our SC. However, we have many good candidates to pick from in the staff of the USACO. I have Rob's assurance that the USACO grading system will be available for our IOI2003. "We have an experienced staff of six problem creators and problem solvers currently at the USACO --Hal Burch, Brian Dean, Greg Galperin, Russ Cox, Reid Barton and Rob. These will be the major players at IOI2003. "We run 5 Internet competitions every year and each one involves more competitors than an IOI. Our US Open involved 447." Don Piele also confirmend in a separate message that IOI2003 would provide competitors with dual-boot Linux-Windows systems for program development. * As in several past ISC meetings, we briefly discussed the choice of programming language(s) available at the IOI competition. We reconfirmed that the current choice of Pascal, C and C++ will be maintained. We feel that all three languages will be needed for at least the next couple of years. Adding Java is still not appropriate, in spite of better Java compilers that produce native code for various platforms. We will keep an eye on how Java is taken up in secondary education. * No new proposals were done concerning the choice of operating system as development environment for the competitors. Dual-boot systems are certainly a burden to the organizers, but the current sentiment in the GA does not swing in one direction. Of course, the GA must then also accept that there is a price for this freedom of development environment. * Concerning an IOI grading system, it was expressed that it would be convenient to have a system that can "travel" with the IOI from year to year. It will be necessary to have some people from the IOI community that document and maintain such a system based on IOI requirements. It was also felt that this might not be so easy to bring about. The notion of an IOI Software Team had already been launched at IOI'99, but it was not followed through. Earlier this week, Kunsoo Park agreed that the system they are developing will be available for future IOI usage, though the copyright will remain with him. He also pointed out, however, that many design decisions were aimed at getting the job done reliably for IOI2002, rather than creating a future-proof system. * This year, three externally-authored tasks were submitted. The smaller number can in part be attributed to lesser advertizing. Two of these submissions made it to the shortlist. External authors must be clearly informed about the status of their submission. If it is not accepted in the final set, then it should be returned to them. On the other hand, if it is accepted into the final set, then they must be requested to maintain confidentiality. The procedure for externally authored tasks is still not sufficiently well defined. This includes the announcement (Call for Tasks), handling of submissions, restrictions that apply to submitters, dealing with unused submissions (e.g. saving them for next year), finalizing all the material to use a task. * No ideas were offered for new task types, though it was expressed that there is still room for further development. Some possible innovations had been discussed earlier this week, such as, floating point scores, "soft" time limits. The grading of competitors' task analysis and design effort, separate from its embodiment in a computer-executable algorithm, still seems out of reach. * The role of the ISC should be evaluated together with the GA. Before the review meeting in Korea, Tom had circulated a list of possible duties for ISC members in the preparation and execution of an IOI competition. Kyung-Yong suggested that the ISC as a whole could be responsible for preparing the competition tasks. Others did not believe that this would work in practice. Linking each task to an ISC member to one or two tasks seems a more workable proposition. The ISC member should study these tasks carefully to understand them in depth and become familiar with their grading (e.g. the test data), in order to act as a liaison between the GA and the Host SC. Candidate ISC members should be clearly informed about ISC duties. Now, the work done by ISC members is often inspired by personal convictions. Tom will polish the document on Roles of ISC Members, and bring it to the attention of the GA. * Jyrki inquired whether the criteria that the IC uses to approve candidate future IOI Hosts were documented anywhere, and in particular, whether this included any scientific and technical requirements concerning the ability to prepare and execute a high-quality competition. For example, is could be required from future Hosts o that they deliver various items according to a pre-agreed schedule, including written plans and progrerss reports, and various details about equipment and support software. o that they have participated in at least five preceding IOIs, o that the chair of their Host SC has attended three previous IOIs and has been a member of the ISC in the preceding year. The IC will be contacted about this. * Djura suggested that the ISC might fight low scores by preparing examplary material for preparing coaches and competitors, rather than by making changes to the competition (such as simpler tasks, partial credit, or more advanced tool support). Some others did not agree that this should be the duty of the ISC. 8. No further discussion items were raised. 9. Tom closes the meeting.