Majeed Kazemitabaar

We developed CodeAid, an LLM-powered programming assistant designed to offer students timely and personalized feedback without directly revealing code solutions. This tool aids in answering conceptual questions, generating pseudo-code, and suggesting corrections for incorrect code. We deployed CodeAid in a large class of 700 students and conducted a thematic analysis of its 8,000 usages, supplemented by weekly surveys and student interviews. Further feedback was obtained from eight programming educators. Results showed that most students used CodeAid for understanding concepts and debugging, though some students directly asked for code solutions. While educators valued its educational merits, they also highlighted concerns about occasional inaccuracies and the potential for students to rely on tools like ChatGPT.

We then concluded with four key design considerations for AI assistants in educational contexts, centered around four main stages of a student's help-seeking process:

• Exploiting Unique Advantages of AI: Decision to use the AI tool, emphasizing the unique advantages of AI over other resources.
• Designing the AI Querying Interface: Query formulation, providing context, and balancing user-friendliness with meta-cognitive engagement.
• Balancing the Directness of AI Responses: Nature of AI responses, managing directness, scaffolding type, and learning engagement.
• Supporting Trust, Transparency and Control: Post-response actions, ensuring accuracy, trust, transparency, and control.

We studied the impact of Large Language Model (LLM)-based code generators, such as OpenAI Codex, on novice programmers (ages 10-17). In a controlled experiment involving 69 novices working on 45 Python tasks, we found that using Codex led to a 1.15x increase in code-authoring completion rate and 1.8x higher scores, without diminishing manual code-modification capabilities. Interestingly, those with prior Codex exposure had slightly improved performance in evaluations a week later. A deeper dive into data from 33 participants who used Codex revealed various ways they interacted with the tool. We identified four coding strategies: AI Single Prompt, AI Step-by-Step, Hybrid, and Manual coding. The AI Single Prompt strategy yielded the highest correctness in code-authoring but struggled in code-modification tasks. Our findings highlighted both the potentials and pitfalls of LLMs in educational settings, emphasizing the need for balanced integration and curriculum development.

We designed CodeStruct, an intermediary programming environment aimed at assisting learners transition from block-based programming, like Scratch, to text-based languages such as Python. CodeStruct bridges the learning curve between these two paradigms, offering design features that significantly reduce completion times and help requests when compared to a direct transition. In a study with 26 high school students, results indicated that those using CodeStruct had a smoother transition with a decrease in data-type and syntax issues, especially when initially aided by a structured editor. However, once they moved to an unstructured editor, the rate of syntax errors increased, though they still outperformed peers who transitioned directly.

A key challenge in developing and debugging custom embedded systems is understanding their behavior, particularly at the boundary between hardware and software. Bifröst automatically instruments and captures the progress of the user's code, variable values, and the electrical and bus activity occurring at the interface between the processor and the circuit it operates in. This data is displayed in a linked visualization that allows navigation through time and program execution, enabling comparisons between variables in code and signals in circuits.

GestureBlocks incorporates a demonstrate-edit-review Machine Learning pipeline for authoring sensor-based gestures into Microsoft MakeCode and allows novices to program behaviors using both data-driven and conventional paradigms.

When programmers face new frameworks they usually rely on example codes to learn about the API and accomplish their tasks. This work investigates and analyzes the activities performed by programmers when such sample codes are being used for task completion.