• How to compare plans for which the time spent is not the same.. ?
• If this is a planning competition then as an organizer you should not accept submission of plan beyond the deadline, as it came to my notice that some planners have submitted multiple plans even after the deadline if they got good results later. As other planners who have stopped planning can also improve their results if it is announced earlier that you are accepting plans even after deadlines too.

• It would be very useful to be able to score my plan online before submitting. It takes significant time to just manually add up the results to compare my iterations.
• My suggestion is to provide participants with IQ plan (demo version, if it is possible) to facilitate the evaluation of plans for competition by themselves. And in my opinion it will be better to give constraints closer to clinical situations. For example, the constraint for right breast was too hard and in clinic we don't have such hard constraints (it is not so important). Organization was at excellent level, it was very pleasant to communicate with Ahmad :-)
• The plan criteria can be summarize a more compact way so its easy for planner to refer.
• Constraints are confusing.. simple Tabular column would be sufficient
• It should be great to have at least an excel file allowing us to know our score during the planning by filling in the dose constraints we manage to get. I don't know if you will give us this information, but it could be very informative to have the anonymized RT plan, RT dose, RT Structures and optimization constraints of the fifth best plans to improve our clinic.

• The organizers should not depend on a single person for the entire process and there should never be a single window approach for any competition, as it questions the faith on the evaluation process.

• I think there should be extra time allocated for the planning because of our everyday clinical work which has to be done first. But you have already asked that ;)
• Unfortunetly I didn't have time enough to finish my planning. The plan became able to be downloaded in a very big holiday in Brazil, so I was traveling. But anyhow it was realy good to participate of that competition. I learned a lot when I was planning and I'll learn still more on the webinars.
• There is usually long pubic hoilday during late Jan to Early Feb for China and Hong Kong region (Lunar New Year) It is difficult to attend competition for Chinese planner.

• Give some absolute limits for couch isocentric rotation (not just saying avoid collision).

Answer:  I think this is difficult.

• We had problem in downloading images from dropbox which is not allowed in UK based clinics. Its really hard to go out of way and get the images into system. Getting time in planning is tough though with heavy workloads. There are other ways to transfer the images or may be it could send across to clinics. Unfortunately, we couldn't participate due to staff availability at the time of competition proposed and internal IT issue of transferring into system. It might be surprising, still the old school exists. Choice adds a flavour.

• We have a good experience of plan challenges in my centre, and we always have the same issue: We would never do in our clinic what we have to do to meet the plan criteria. This includes: - Tweaking the machine model to improve gradients, etc. - Spending the two weeks almost full time on the plan challenge - Manually moving MLC leaves - Making a very complex plan (too many arcs or beams, too many segments) - Making a plan we know would not pass our patient-specific QA test For this challenge, I produced a real plan. We may not threat breasts in VMAT with 2 partial arcs, but I am fairly confident my plan would pass QA test. I did not tweak the model and did not relax and planning constrain parameter. In my clinic, we always ask ourselves jokingly "Would you treat your mother with this plan?" Here, I can say with confidence "yes, I would." You say in your email, that the best score is 98.2%; I really doubt that plan would be clinically deliverable.
• Unfortunately, with clinical cases and in house research on-going, time for any extra activity if not planned in advance is limited and therefore little time was given to the competition. This did not allow me to explore new ways of "adjusting constraints" in the system. Nevertheless though I say I did not improve my skills it has been beneficial in terms of internal discussions between colleagues and for introducing the idea of new planning procedures. Thanks

Answer:  Thank you for your time and nice comments. The experience gained through the days to achieve a great plan quality will teach planner workarounds that will surely help them in future cases, and will try to produce also “practical and deliverable” plans as well. Hopefully next time you will have enough time as the coming planning time allocated will be 3 weeks to submit your final plan.

• I didn't hear about the challenge till 2 days before the deadline. I would have loved to spend more time on this but with clinical duties could not. Many colleagues of mine would have done a plan too if we heard about the event sooner. I never saw the event advertised, so maybe send out some announcements for the USA. In general I enjoy doing these competitions and the interactions with Ahmad were helpful and he responded very quickly to questions, even though he was on the other side of the world.
• More advertising of the competition. In order to save printed paper, a summary with VOI limita and máximum punctuation if achieved.
• I did know about the competition late, so advertising it more would be great!
• I'm not sure if it requires a modification but I did not hear about the competition until late, around 1 week or so before the deadline. If there's any way to get the word out earlier, that would be helpful.
• To inform physicists of the existence of the next competitions, it might be judicious to go through the national medical physics societies (for exemple, NCS for Netherlands).
• Except the time required to Plan. All other things are wonderfully done.

Answer:  Thank you for your time and nice comment. The experience gained through the days to achieve a great plan quality will teach planner workarounds that will surely help them in future. I tried my best to advertise it, we used physics list, dosimetry list, linkedin, facebook, even we contacted medical physics societies in different countries, but we promise next time to announce it in a better way J

• Turning it into a competition with a final, semi final,... But use different localisation to show we can be good on various sites and tumors, to show adaptability to situations indicate the prescription of sites and doses

• Actually I have problem in sending plan I hope you have time to tell us about properties of folder that contain plan and ct Best regards
• Unfortunately our hospital has blocked cloud services like dropbox, so in the future it could be difficult for us to participate without a plan B.
• greater allocation of evalution point return transfer with "wetransfer" and no drupbox
• I dont know how will be the evaluation. But I think u can have ranks by planning system, machines.... I Think that website must be improved. User friendly.

• Congratulations, continue your Good work

• Make categories according to the irradiation technique (IMRT - VMAT - tomo...)
• Machine limitations and/or the TPS' physical machine model will strongly influence results. Maybe results should be compared/grouped by accelerator class and not only by treatment technique types.
• plan evaluation criteria should also depend upon the linear accelerator on which plan is being made as size of MLC, machine head properties also degrade the plan quality. Different TPS's have different limitation for the planners so evaluating the plan by neglecting these points is not fair.
• I think it will be better if all planners have the same linac configuration in order to really compare the skill to make plans.

• Competition results should be announced as earlier as possible

• If possible, accept hybrid plans (i.e 3D+Imrt... ) in the evaluation.

• Perhaps have a stricter planning guideline. e.g. one competition to only use 3DCRT and one to only use inverse technique. This way skill can be tested, not the planning system level.

• Competition is well organised, I would like to suggest to follow the planned dates like, start date, completion date, evaluation date & result day. I understand this is your first ever competition organized, but this is a suggestion.

• When trying to achieve a point here or there, one sometimes loses the clinical picture. It is important to stress that. I think the metric values were a little mismatched here. There should be min goals to be accepted. In this case a 70 could be achieved by doing nothing.

• The focus of the competition should be on creating real clinical plans and not idealistied plans which no one would ever irradiate. I thing this should be a must criteria for the next competition.

• The points of the objectives could be more realistic in terms of the clinical importance of each organ at risk. For example, lung or heart should have many more points than contralateral breast, on this case.

• Dont compare the plan results with tomo, cyberknife with LINAC. Which done was a (***) results

• More notice for the live webinar would be beneficial to ensure people can organise themselves to participate. Approximately 1 week would be good. Thank you.

• Plan techniques much important for overall result. So try to compare same technique plans

• I was a bit disapointed that i can't send my plan because i did not receive the link to upload my plan. I have send you an e-mail on this problem but i neveu receive a feed-back...

• It would be better to get all information about the next competition, about files we need to upload and everything else earlier. To make it easier to transfer and communicate.

• I think that the competition need more publicity to improve number of participants and increase level of results. Maybe the solution is to present this competition on mains physicist's congresses.

• The informative webinars should be recorded and made easily available . That is the learning process and is the most beneficial part of this competition but I found it dificult to attend them due to different time zone. Please provide access to the recorded webinars of the 3 d plan winner and other plan winners

• It was great experience Looking next competition Thank you very much

Answer:  It is our pleasure to have you and all other planners among us