HomeGitHub Copilot Translates Mockups Into Code with Image-to-Code Generation
GitHub has unveiled a series of updates for Copilot, offering a preview of a more autonomous future for its AI-powered programming assistant.
A key new feature is Copilot Vision, which lets users upload a screenshot, photo, or diagram directly into the chat. Copilot then uses the visual input to generate corresponding interface elements, code, or descriptive alt text.
For instance, a marketing team member could capture a webpage screenshot and visually indicate desired changes. Instead of describing the edits in text, they can simply upload the annotated image and instruct Copilot to implement the shown modifications.
It's important to note that this functionality was previously available as a VS Code extension starting last October. Microsoft had indicated it would eventually be deprecated in favor of a native feature within GitHub Copilot Chat. The earlier version also required users to supply their own API key.
Vision for Copilot | Image Credits: GitHub GitHub is also launching "next edit suggestions," an enhancement to Copilot's core code completion capabilities.
Previously, GitHub Copilot in the VS Code editor primarily worked from the cursor's current position. Now, it analyzes recent edits across the file to predict the developer's likely next action, using richer contextual signals to automatically propose the subsequent edit.
Developers can accept these suggestions by pressing the Tab key or reject them by pressing Esc.
GitHub Copilot Next Edit Suggestions | Image Credits: GitHub GitHub's Autonomous Future
GitHub first introduced Copilot Edits last November, providing a natural language interface for executing multi-file changes. This feature is now generally available and includes a new "agent mode." This mode automatically identifies all files relevant to the developer's requested changes, eliminating the need to manually specify which files should be modified.
"Copilot now does more work to understand the intent behind your original request and then tries to solve it," GitHub CEO Thomas Dohmke explained to TechCrunch.
Long-term, the vision is to apply similar "agent modes" to other areas of Copilot, enabling greater automation across interconnected development tasks. This direction is underscored by a new initiative called Project Padawan, essentially a software engineering (SWE) agent capable of independently handling entire tasks when a developer assigns an issue to Copilot.
Dohmke did not provide a specific timeline for Project Padawan's full release, stating only that GitHub will collaborate with partners and the community to refine it.
"We're seeking partners who are also building agents to integrate into that workflow, and customers to work with us and provide feedback—because we know it won't be perfect at this stage," Dohmke said.
TechCrunch publishes a dedicated AI newsletter. Sign up here to receive it in your inbox every Wednesday.
Related article
Microsoft retires failed AI tools and consolidates Copilot apps
Two years ago, Microsoft characterized AI as a “generational shift” in technology that it aimed to lead. Today, the company is consolidating its Copilot-branded consumer and business applications, while discontinuing several underperforming AI featur
Haleon and Microsoft Partner to Automate Global Health
Haleon is leveraging Microsoft AI to accelerate scientific discoveries in everyday health, including dental care. (Credit: Haleon)The UK-based consumer health firm Haleon has entered a partnership with Microsoft, a step that Claire Dickson, its Chief
Microsoft pulls back Copilot AI bloat on Windows
On Friday, Microsoft announced a series of updates aimed at enhancing the quality of its Windows 11 operating system, most notably reducing the number of access points for its AI assistant, Copilot.The company stated that it will scale back Copilot A
Related Special Topic Recommendations
Image editing
Photoshop AI Retouch Tools for Ecommerce Apparel, Skin Cleanup, and Color Consistency
2026 Latest Best Photoshop AI retouch tools for ecommerce apparel, skin cleanup, and color consistency! This top-rated curated list features powerful game-changing solutions that help you boost writing efficiency, streamline content creation, and achieve perfect visual results effortlessly. Each tool has undergone real-world tests through weekly updated rankings, complete with free vs paid comparison details. Backed by XIX.AI, it’s the must-try guide for anyone aiming to unlock your AI edge. Explore now!
10 tools
xix.ai
Prompt
Best AI Prompt Libraries for ChatGPT Workflows
2026 Latest Best Top-Rated AI Prompt Libraries for optimizing all types of ChatGPT workflows. XIX.AI has curated a powerful, game-changing collection that goes through rigorous real-world tests to ensure top performance. You can find detailed free vs paid comparisons and expert rankings to help you choose the must-try tools that boost your productivity and unlock your AI edge. Explore now!
11 tools
xix.ai
Education and Learning
AI Quiz Builder Platforms for Teachers, Tutors, and Cohort-Based Learning Programs
2026 Latest Best AI Quiz Builder Platforms for Teachers, Tutors, and Cohort-Based Learning Programs! XIX.AI has curated a top-rated list of powerful game-changing tools that go through real-world tests to deliver accurate rankings. These must-try platforms help boost writing efficiency, streamline content creation, and simplify quiz design across all learning scenarios. Explore now to discover your perfect tool for unlocking your AI edge in teaching!
13 tools
xix.ai
code
AI Pull Request Review Tools for GitHub Teams Handling Refactors, Bugs, and Security Gaps
2026 Latest Best AI Pull Request Review Tools for GitHub Teams are here on XIX.AI! This top-rated curated list showcases powerful game-changing solutions that streamline refactoring, bug fixing, and security gap detection across all team workflows. Enjoy a free vs paid comparison along with real-world tests and detailed rankings to help you find the perfect tool that boosts productivity significantly. Explore now to unlock your AI edge!
12 tools
xix.ai
Text-to-speech
Best AI Text to Speech Tools for Natural Voiceovers
2026 Latest Best Top-rated AI Text to Speech Tools for Natural Voiceovers are here on XIX.AI! This curated list features powerful, game-changing options that deliver crystal-clear voices for every use case, backed by real-world tests and weekly updated rankings. Get a free vs paid comparison to find the must-try solution that boosts your productivity instantly. Explore now to Unlock your AI edge!
11 tools
xix.ai
Comic Creation
Manga AI Background Generators for Serialized Chapters, Covers, and Promo Art
2026 Latest Best Manga AI Background Generators Ranked Top-Rated! This curated collection showcases powerful game-changing tools perfect for creating high-quality chapter backgrounds, book covers, and promotional art. Every option has undergone rigorous real-world tests to ensure reliability. Get a free vs paid comparison along with detailed insights. Explore now to discover your perfect tool and unlock your AI edge in manga creation.
6 tools
xix.ai
Comments (2)
0/500
![JonathanGreen]()
Ah, l'idée de transformer une simple capture d'écran en code via Copilot, c'est séduisant sur le papier, mais franchement, je me demande si ça va vraiment marcher sans générer un tas de bugs invisibles. On dirait un gadget de plus pour faire joli dans les démos, mais dans la vraie vie, le diable se cache dans les détails. Peur que ça finisse par produire du code spaghetti que personne ne voudra relire. 😅
![FrankSanchez]()
This is a game-changer for prototyping! Just imagine snapping a quick sketch and having functional code pop out. It feels like we're inching closer to that 'idea to app' dream, though I wonder how it handles more complex UI logic beyond basic layouts. The potential for rapid iteration is huge.
GitHub has unveiled a series of updates for Copilot, offering a preview of a more autonomous future for its AI-powered programming assistant.
A key new feature is Copilot Vision, which lets users upload a screenshot, photo, or diagram directly into the chat. Copilot then uses the visual input to generate corresponding interface elements, code, or descriptive alt text.
For instance, a marketing team member could capture a webpage screenshot and visually indicate desired changes. Instead of describing the edits in text, they can simply upload the annotated image and instruct Copilot to implement the shown modifications.
It's important to note that this functionality was previously available as a VS Code extension starting last October. Microsoft had indicated it would eventually be deprecated in favor of a native feature within GitHub Copilot Chat. The earlier version also required users to supply their own API key.
GitHub is also launching "next edit suggestions," an enhancement to Copilot's core code completion capabilities.
Previously, GitHub Copilot in the VS Code editor primarily worked from the cursor's current position. Now, it analyzes recent edits across the file to predict the developer's likely next action, using richer contextual signals to automatically propose the subsequent edit.
Developers can accept these suggestions by pressing the Tab key or reject them by pressing Esc.
GitHub's Autonomous Future
GitHub first introduced Copilot Edits last November, providing a natural language interface for executing multi-file changes. This feature is now generally available and includes a new "agent mode." This mode automatically identifies all files relevant to the developer's requested changes, eliminating the need to manually specify which files should be modified.
"Copilot now does more work to understand the intent behind your original request and then tries to solve it," GitHub CEO Thomas Dohmke explained to TechCrunch.
Long-term, the vision is to apply similar "agent modes" to other areas of Copilot, enabling greater automation across interconnected development tasks. This direction is underscored by a new initiative called Project Padawan, essentially a software engineering (SWE) agent capable of independently handling entire tasks when a developer assigns an issue to Copilot.
Dohmke did not provide a specific timeline for Project Padawan's full release, stating only that GitHub will collaborate with partners and the community to refine it.
"We're seeking partners who are also building agents to integrate into that workflow, and customers to work with us and provide feedback—because we know it won't be perfect at this stage," Dohmke said.
TechCrunch publishes a dedicated AI newsletter. Sign up here to receive it in your inbox every Wednesday.
Microsoft retires failed AI tools and consolidates Copilot apps
Two years ago, Microsoft characterized AI as a “generational shift” in technology that it aimed to lead. Today, the company is consolidating its Copilot-branded consumer and business applications, while discontinuing several underperforming AI featur
Haleon and Microsoft Partner to Automate Global Health
Haleon is leveraging Microsoft AI to accelerate scientific discoveries in everyday health, including dental care. (Credit: Haleon)The UK-based consumer health firm Haleon has entered a partnership with Microsoft, a step that Claire Dickson, its Chief
Microsoft pulls back Copilot AI bloat on Windows
On Friday, Microsoft announced a series of updates aimed at enhancing the quality of its Windows 11 operating system, most notably reducing the number of access points for its AI assistant, Copilot.The company stated that it will scale back Copilot A
2026 Latest Best Photoshop AI retouch tools for ecommerce apparel, skin cleanup, and color consistency! This top-rated curated list features powerful game-changing solutions that help you boost writing efficiency, streamline content creation, and achieve perfect visual results effortlessly. Each tool has undergone real-world tests through weekly updated rankings, complete with free vs paid comparison details. Backed by XIX.AI, it’s the must-try guide for anyone aiming to unlock your AI edge. Explore now!
10 tools
xix.ai
2026 Latest Best Top-Rated AI Prompt Libraries for optimizing all types of ChatGPT workflows. XIX.AI has curated a powerful, game-changing collection that goes through rigorous real-world tests to ensure top performance. You can find detailed free vs paid comparisons and expert rankings to help you choose the must-try tools that boost your productivity and unlock your AI edge. Explore now!
11 tools
xix.ai
2026 Latest Best AI Quiz Builder Platforms for Teachers, Tutors, and Cohort-Based Learning Programs! XIX.AI has curated a top-rated list of powerful game-changing tools that go through real-world tests to deliver accurate rankings. These must-try platforms help boost writing efficiency, streamline content creation, and simplify quiz design across all learning scenarios. Explore now to discover your perfect tool for unlocking your AI edge in teaching!
13 tools
xix.ai
2026 Latest Best AI Pull Request Review Tools for GitHub Teams are here on XIX.AI! This top-rated curated list showcases powerful game-changing solutions that streamline refactoring, bug fixing, and security gap detection across all team workflows. Enjoy a free vs paid comparison along with real-world tests and detailed rankings to help you find the perfect tool that boosts productivity significantly. Explore now to unlock your AI edge!
12 tools
xix.ai
2026 Latest Best Top-rated AI Text to Speech Tools for Natural Voiceovers are here on XIX.AI! This curated list features powerful, game-changing options that deliver crystal-clear voices for every use case, backed by real-world tests and weekly updated rankings. Get a free vs paid comparison to find the must-try solution that boosts your productivity instantly. Explore now to Unlock your AI edge!
11 tools
xix.ai
2026 Latest Best Manga AI Background Generators Ranked Top-Rated! This curated collection showcases powerful game-changing tools perfect for creating high-quality chapter backgrounds, book covers, and promotional art. Every option has undergone rigorous real-world tests to ensure reliability. Get a free vs paid comparison along with detailed insights. Explore now to discover your perfect tool and unlock your AI edge in manga creation.
6 tools
xix.ai
Ah, l'idée de transformer une simple capture d'écran en code via Copilot, c'est séduisant sur le papier, mais franchement, je me demande si ça va vraiment marcher sans générer un tas de bugs invisibles. On dirait un gadget de plus pour faire joli dans les démos, mais dans la vraie vie, le diable se cache dans les détails. Peur que ça finisse par produire du code spaghetti que personne ne voudra relire. 😅
This is a game-changer for prototyping! Just imagine snapping a quick sketch and having functional code pop out. It feels like we're inching closer to that 'idea to app' dream, though I wonder how it handles more complex UI logic beyond basic layouts. The potential for rapid iteration is huge.
