The research evaluates the communication competence of LLMs in code generation tasks, highlighting the significance of asking clarifying questions. The study introduces a new benchmark, HumanEvalComm, to assess LLMs’ ability to reduce ambiguity in problem descriptions. Metrics like Communication Rate and Good Question Rate are defined to measure LLMs’ communication skills. The findings suggest ways to enhance LLMs’ code generation efficiency by asking relevant questions. - Evaluation of LLMs’ communication skills for code generation. - Introduction of HumanEvalComm benchmark for LLMs. - Metrics like Communication Rate and Good Question Rate defined. - Comparison with new LLM agent approach Okanagan. The paper sheds light on improving LLMs’ code generation efficiency by emphasizing the importance of communication skills and asking clarifying questions in problem-solving scenarios.