Benchmarking the Communication Competence of Code Generation for LLMs and LLM Agent

Large language models (LLMs) have significantly improved their ability to perform tasks in code generation. However, a gap exists in communication skills necessary for top-tier software engineering. This empirical study assesses the communication abilities of LLMs for code generation tasks… Read more

• Introduction of HumanEvalComm benchmark for communication skills
• Evaluation metrics including Communication Rate and Good Question Rate
• Comparison of Code LLMs and Okanagan LLM agent in refining code

This research highlights the importance of effective communication in improving the code generation capabilities of LLMs and LLM agents.