Minutes: International Scientific Committee Meeting 5 Time/Date: 09:15 -- 11:10, 20 July 2001 Place: Tampere Hall, Tampere, Finland Status: Approved (September 2001) Present (alphabetical order): Hal Burch Host n+2 (new member) Kyung-Yong Chwa Host n+1 Gyula Horvath Elected (left after point 4) Ian Munro Elected Jyrki Nummenmaa Host n (delayed) Kunsoo Park Host n+1 (new member) Djura Paunic Elected (new member) Tom Verhoeff Elected (chair) Baolin Yin Host n-1 (left after point 4) Absent: Isto Aho Host n (leaving member) 1. The meeting is opened by Tom Verhoeff. 2. The agenda is approved. 3. It is agreed that Tom chairs this meeting. 4. Tom thanks the leaving members on behalf of everyone for their invaluable contributions to the effectiveness of the committee and to the development of the IOI in general. Gyula expresses that he enjoyed working on the committee. He is convinced that scientifically the competition is at a good level. However, he feels that the implementation needs attention. Especially, he finds it important to improve the efficiency of the ISC. Hard work was done, but too much time was wasted in the end. He also emphasized that the "Guidelines for IOI Competitions", which the ISC approved before IOI2000, must be applied more carefully in the future. Finally, he urged that the ISC specifies the requirements for IOI grading systems, including the various interfaces. Yin thanks the committee for the fruitful cooperation. He expresses agreement with Gyula's points, adding the following. He finds that the ISC must be "master" of the grading system, which should preferably be reusable and adaptable by others in the form of open source software. He suggests to include further details in the Guidelines, e.g. concerning the procedure for approving the medal boundaries, and the working procedure of the ISC, especially during an IOI. He also recommends that a way be sought to better enforce deadlines. Finally, he explained that using two operating systems is not a good idea, and that both from a technical and a business perspective it will be better to use Linux only. New members are welcomed, and requested to provide contact information: Kunsoo Park (Host IOI2002), kpark@theory.snu.ac.kr Hal Burch (Host IOI2003), hburch@lumeta.com Djura Paunic (elected), djura@unsim.im.ns.ac.yu The ISC's role is reconfirmed. The official status of the "Guidelines for IOI Competitions" is unchanged since last year. Neither IC nor GA have ever formally approved them. At IOI2000, the IC decided that the Guidelines were too "detailed/complex" to put forward to the GA for approval. The IC then promised to express to the GA at IOI2000 their support for the principles set out in the Guidelines. However, the IC has never actually done this. It must be noted that the ISC itself has approved the Guidelines for use as their "published operating principles". 5. It is re-emphasized that members of the ISC may at some point receive information whose disclosure might damage the success of an IOI competition. All ISC members confirm that they understand that such information must be handled with utmost care. For this reason, it is asked that ISC members refrain from coaching and training involvement related to IOI2002 participation. A second reason to refrain from such training involvement has to do with the credability of the ISC. Even though we may know how to protect sensitive information while coaching, it might reflect badly on the ISC when its members are involved in coaching. Over the past two years, the ISC has successfully used PGP encryption to to protect confidential information in e-mail messages. It is decided to continue this practice. New members are requested to provide their PGP public keys to Tom, who publish them on the website. 6. One round was made, where those present in the meeting expressed their opinion on the IOI2001 competition that was just experienced. Ian: Getting things done early needs improvement. Jyrki has, to some extent understandably, been on the critical path of almost everything. This also leads to more stress and subsequently to unnecessary mistakes. How can that be reduced? Communication has been a bottleneck and needs attention. The grading system used at IOI2001 was a comprehensive piece of software, and the IOI community should express its gratitude to Rob Kolstad, who has been mostly invisible because of all the work that he needed to do. Jyrki responds that Rob will receive a gift in the Closing Ceremony later today. Hal: The practice competition was a required item, especially this year with all the changes. The after-grading process where the delegations inspect the results needs further attention. Concerning the grading system, it must be noted that the "test button" brings back batch debugging of 40 years ago. Is that the way we want to go? The end-of-contest delays should be reduced, and the delays in producing the grading results must be analyzed to help eliminate them. Djura: The wording of tasks can be done more uniformly. It may be better to eliminate the Task Overview sheet and include that information in the task it concerns. During the Practice Competition, a stress test could be done by coordinating the simultaneous submission of material by all competitors, say at the blow of a whistle. Kyung-Yong: Information could be shared more openly with the GA. Supports Ian's comments. Kunsoo: Chairs the IOI2002 grading committee, consisting of 3 or 4 persons. The use of two operating systems introduces many complications. He would like to provide only Windows, since that is what the majority of the competitors seemed to be using. Hal points out that the USA, Host for IOI2003, intends to have a Linux-only competition environment. Finally, he had the impression that Rob carried too much responsibility in the grading process. Without him it would have failed completely (single-point failure). Jyrki: Although external submissions bring diversity, it is difficult to do proper quality control. There is no penalty when the task author fails to adhere to agreed deadlines. Tom notes that we discussed this in May, and there is a proposal to streamline the process. The Practice Competition may have been useful, but it also took away scarce resources needed for preparing the actual competition days. It should be noted that the local Host SC was very small. Besides Jyrki, it consisted of: Teppo, Isto, Timo T., Timo P., Tero, Samuli, and remotely also Rob Kolstad. Configuring the development environments for the competitors was much harder than expected, both in terms of software and hardware. For example, the need to make available the same versions of the compilers under both Linux and Windows posed severe restrictions. It eliminated Red Hat 7, because that is based on gcc version 2.96.x, etc. The orinally selected video card was too new for the drivers of the X Windowing system. The Linux installation on the competitor machines was not only for the competitors. It was also used to manage the Windows installation, whose hard disk could be mounted and managed under Linux, for example, to distribute files to the competitor computers before each Competition Day. The Windows environment was a major cause of the complications. For example, to ensure secure communications, special switches had to be used (being managed from Linux). There were some surprises for the grading system, such as program sources exceeding one million lines of code. In one case, the source was submitted in rapid succession, each time a substantial number of lines. This caused excessive compile times and reduced the performance of the system. Jyrki agrees to transfer the IOI2001 competition material to the IOI2002 Host SC before mid September. Tom: Suggests that IOI2002 SC prepares a report about their experiences at the IOI2001 Competition. This will help understand and overcome many of the difficulties. It is recommended to prepare statistics about the (intermediate) scores more systematically and discuss them in the ISC, before making decisions and presenting results to the delegations or the GA. 7. It is agreed that the Guidelines for IOI Competitions will be applied to the preparation and execution of the IOI2002 Competition, possibly after making some amendments. It is also possible to document separately which items in the Guidelines will be treated differently, and if so, how. In particular, it is necessary to start early with project planning and progress reporting. May 2002 is proposed by the IOI2002 SC as the period for the ISC review meeting, because end of May (30 or 31), the World Cup Soccer is held in Korea. Jyrki is not available in the first week of May. Djura prefers early May, because the Balkan Olympiad in Informatics will be held in May 2002 in Yugoslavia (dates not yet determined). Tom will take initiative to select a date for the review. Kyung-Yong tells that the IOI2002 SC, grading committee, and technical committee have already been meeting on a monthly basis, and that they are collecting information in a database. The information is mostly in Korean. Tom request to prepare a description of the contents in English for the ISC. Kyung-Yong explains that a Call for Tasks had been prepared for release during IOI2002, but that it was decided to wait. He will send it to the ISC first. The proposed deadline for submission is end of December 2001. Kunsoo asks how they can get the source of the grading system used at IOI2001. Jyrki points out that it is Rob's product, copyrighted "Rob Kolstad & IOI", since the IOI2001 SC also contributed pieces. Rob has the source; documentation is limited and in many ways out-of-date due to the many changes that were made in the last couple of months. Hal suggests to wait at least one month before approaching Rob. 8. The members suggest and approve that Tom chairs the ISC for another year. Tom points out that it is his last year (unless he is nominated and re-elected), and that it may not be wise to hand over the chair position only at the end of IOI2002. Jyrki proposes that it may be possible to change chair halfway, if it becomes clear that the need arises. No further issues. 9. Tom closes the meeting. Action List ------ ---- All: Check agenda for restrictions on a May 2002 ISC review meeting in May 2002 in Korea. If necessary, set up a PGP key pair, and submit public key to Tom. Deadline: end August. Tom: Collect PGP public keys of new members, update ISC web page. Coordinate selection of date for ISC Review Meeting in Korea. Deadline: mid September. Process the returned questionnaires and summarize the results. Deadline: end September. Jyrki: Provide IOI2001 competition material to IOI2002 SC. Deadline: mid September, before his teaching term starts. Kyung-Yong: Consider production of IOI2001 report, from the point of view of the IOI2002 SC. Go through "Guidelines for IOI Competitions" and propose how to apply them to IOI2002. Prepare description of database for ISC. Submit "Call for Tasks" to ISC for review. Open items: Specify requirements for IOI Grading System. Update the "Guidelines for IOI Competitions". Determine roles of ISC members during an IOI. Streamline process of external task submissions. And more...