You've already forked Nano-Banana-AI-Image-Editor
修复 快捷键无效的问题;
修复 重复上传参考图的问题; 重新编写了README文档;
This commit is contained in:
20
README.md
20
README.md
@@ -1,5 +1,6 @@
|
|||||||
# 🍌 Nano Banana AI 图像编辑器
|
# 🍌 Nano Banana AI 图像编辑器
|
||||||
发布版本: (v1.0)
|
|
||||||
|
发布版本: v1.0
|
||||||
|
|
||||||
### **⏬ 获取一键安装副本!**
|
### **⏬ 获取一键安装副本!**
|
||||||
加入 [Vibe Coding is Life Skool 社区](https://www.skool.com/vibe-coding-is-life/about?ref=456537abaf37491cbcc6976f3c26af41) 获取此应用的 **一键 ⚡Bolt.new 安装克隆**,以及现场构建会话、独家项目下载、AI 提示、大师课程和网络上最好的氛围编码社区的访问权限!
|
加入 [Vibe Coding is Life Skool 社区](https://www.skool.com/vibe-coding-is-life/about?ref=456537abaf37491cbcc6976f3c26af41) 获取此应用的 **一键 ⚡Bolt.new 安装克隆**,以及现场构建会话、独家项目下载、AI 提示、大师课程和网络上最好的氛围编码社区的访问权限!
|
||||||
@@ -10,10 +11,6 @@
|
|||||||
|
|
||||||
一个生产就绪的 React + TypeScript 应用程序,用于愉快的图像生成和使用 Google Gemini 2.5 Flash Image 模型进行对话式、区域感知的修改。采用现代网络技术构建,专为创作者和开发者设计。
|
一个生产就绪的 React + TypeScript 应用程序,用于愉快的图像生成和使用 Google Gemini 2.5 Flash Image 模型进行对话式、区域感知的修改。采用现代网络技术构建,专为创作者和开发者设计。
|
||||||
|
|
||||||
[](https://nanobananaeditor.dev)
|
|
||||||
|
|
||||||
🍌 [试用在线演示](https://nanobananaeditor.dev)
|
|
||||||
|
|
||||||
## ✨ 主要功能
|
## ✨ 主要功能
|
||||||
|
|
||||||
### 🎨 **AI 驱动的创作**
|
### 🎨 **AI 驱动的创作**
|
||||||
@@ -41,7 +38,7 @@
|
|||||||
- **资产管理** - 有序存储所有生成的内容
|
- **资产管理** - 有序存储所有生成的内容
|
||||||
|
|
||||||
### 🔒 **企业功能**
|
### 🔒 **企业功能**
|
||||||
- **SynthID 水印** - 内置 AI 来源追踪和隐形水印
|
- **图像上传和分享** - 上传生成的图像以轻松分享
|
||||||
- **离线缓存** - IndexedDB 存储以实现离线资产访问
|
- **离线缓存** - IndexedDB 存储以实现离线资产访问
|
||||||
- **类型安全** - 完整的 TypeScript 实现和严格类型检查
|
- **类型安全** - 完整的 TypeScript 实现和严格类型检查
|
||||||
- **性能优化** - React Query 实现高效状态管理
|
- **性能优化** - React Query 实现高效状态管理
|
||||||
@@ -51,6 +48,7 @@
|
|||||||
### 先决条件
|
### 先决条件
|
||||||
- Node.js 18+
|
- Node.js 18+
|
||||||
- 一个 [Google AI Studio](https://aistudio.google.com/) API 密钥
|
- 一个 [Google AI Studio](https://aistudio.google.com/) API 密钥
|
||||||
|
- 可选:访问令牌用于图像上传功能
|
||||||
|
|
||||||
### 安装
|
### 安装
|
||||||
|
|
||||||
@@ -65,6 +63,7 @@
|
|||||||
```bash
|
```bash
|
||||||
cp .env.example .env
|
cp .env.example .env
|
||||||
# 将您的 Gemini API 密钥添加到 VITE_GEMINI_API_KEY
|
# 将您的 Gemini API 密钥添加到 VITE_GEMINI_API_KEY
|
||||||
|
# 可选:添加访问令牌到 VITE_ACCESS_TOKEN 以启用图像上传
|
||||||
```
|
```
|
||||||
|
|
||||||
3. **启动开发服务器**:
|
3. **启动开发服务器**:
|
||||||
@@ -101,12 +100,14 @@
|
|||||||
| 快捷键 | 操作 |
|
| 快捷键 | 操作 |
|
||||||
|----------|--------|
|
|----------|--------|
|
||||||
| `Cmd/Ctrl + Enter` | 生成/应用编辑 |
|
| `Cmd/Ctrl + Enter` | 生成/应用编辑 |
|
||||||
|
| `Enter` | 生成/应用编辑(在任何地方按下)|
|
||||||
| `Shift + R` | 重新生成变体 |
|
| `Shift + R` | 重新生成变体 |
|
||||||
| `E` | 切换到编辑模式 |
|
| `E` | 切换到编辑模式 |
|
||||||
| `G` | 切换到生成模式 |
|
| `G` | 切换到生成模式 |
|
||||||
| `M` | 切换到选择模式 |
|
| `M` | 切换到选择模式 |
|
||||||
| `H` | 切换历史面板 |
|
| `H` | 切换历史面板 |
|
||||||
| `P` | 切换提示面板 |
|
| `P` | 切换提示面板 |
|
||||||
|
| `Esc` | 中断生成 |
|
||||||
|
|
||||||
## 🏗️ 架构
|
## 🏗️ 架构
|
||||||
|
|
||||||
@@ -130,6 +131,7 @@ src/
|
|||||||
│ └── InfoModal.tsx # 关于模态框和链接
|
│ └── InfoModal.tsx # 关于模态框和链接
|
||||||
├── services/ # 外部服务集成
|
├── services/ # 外部服务集成
|
||||||
│ ├── geminiService.ts # Gemini API 客户端
|
│ ├── geminiService.ts # Gemini API 客户端
|
||||||
|
│ ├── uploadService.ts # 图像上传服务
|
||||||
│ ├── cacheService.ts # IndexedDB 缓存层
|
│ ├── cacheService.ts # IndexedDB 缓存层
|
||||||
│ └── imageProcessing.ts # 图像处理工具
|
│ └── imageProcessing.ts # 图像处理工具
|
||||||
├── store/ # Zustand 状态管理
|
├── store/ # Zustand 状态管理
|
||||||
@@ -149,11 +151,13 @@ src/
|
|||||||
### 环境变量
|
### 环境变量
|
||||||
```bash
|
```bash
|
||||||
VITE_GEMINI_API_KEY=your_gemini_api_key_here
|
VITE_GEMINI_API_KEY=your_gemini_api_key_here
|
||||||
|
VITE_ACCESS_TOKEN=your_access_token_here # 可选,用于图像上传
|
||||||
|
VITE_UPLOAD_ASSET_URL=your_asset_url # 可选,用于图像上传的资产URL前缀
|
||||||
```
|
```
|
||||||
|
|
||||||
### 模型配置
|
### 模型配置
|
||||||
- **模型**: `gemini-2.5-flash-image-preview`
|
- **模型**: `gemini-2.5-flash-image-preview`
|
||||||
- **输出格式**: 1024×1024 PNG 带 SynthID 水印
|
- **输出格式**: 1024×1024 PNG
|
||||||
- **输入格式**: PNG, JPEG, WebP
|
- **输入格式**: PNG, JPEG, WebP
|
||||||
- **温度范围**: 0-1 (0 = 确定性, 1 = 创意)
|
- **温度范围**: 0-1 (0 = 确定性, 1 = 创意)
|
||||||
|
|
||||||
@@ -223,4 +227,4 @@ npm run lint # 运行 ESLint
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
**由 [Mark Fulton](https://markfulton.com) 构建** | **由 Gemini 2.5 Flash Image 提供支持** | **使用 Bolt.new 制作**
|
**由 [Mark Fulton](https://markfulton.com) 构建** | **由 Gemini 2.5 Flash Image 提供支持** | **使用 Bolt.new 制作**
|
||||||
@@ -51,15 +51,15 @@ export const useImageGeneration = () => {
|
|||||||
// 上传生成的图像和参考图像
|
// 上传生成的图像和参考图像
|
||||||
if (accessToken) {
|
if (accessToken) {
|
||||||
try {
|
try {
|
||||||
// 上传生成的图像
|
// 上传生成的图像(跳过缓存,因为这些是新生成的图像)
|
||||||
const imageUrls = outputAssets.map(asset => asset.url);
|
const imageUrls = outputAssets.map(asset => asset.url);
|
||||||
const outputUploadResults = await uploadImages(imageUrls, accessToken);
|
const outputUploadResults = await uploadImages(imageUrls, accessToken, true);
|
||||||
|
|
||||||
// 上传参考图像(如果存在)
|
// 上传参考图像(如果存在,使用缓存机制)
|
||||||
let referenceUploadResults: any[] = [];
|
let referenceUploadResults: any[] = [];
|
||||||
if (request.referenceImages && request.referenceImages.length > 0) {
|
if (request.referenceImages && request.referenceImages.length > 0) {
|
||||||
const referenceUrls = request.referenceImages.map(img => `data:image/png;base64,${img}`);
|
const referenceUrls = request.referenceImages.map(img => `data:image/png;base64,${img}`);
|
||||||
referenceUploadResults = await uploadImages(referenceUrls, accessToken);
|
referenceUploadResults = await uploadImages(referenceUrls, accessToken, false);
|
||||||
}
|
}
|
||||||
|
|
||||||
// 合并上传结果
|
// 合并上传结果
|
||||||
@@ -300,7 +300,8 @@ export const useImageEditing = () => {
|
|||||||
if (accessToken) {
|
if (accessToken) {
|
||||||
try {
|
try {
|
||||||
const imageUrls = outputAssets.map(asset => asset.url);
|
const imageUrls = outputAssets.map(asset => asset.url);
|
||||||
uploadResults = await uploadImages(imageUrls, accessToken);
|
// 上传编辑后的图像(跳过缓存,因为这些是新生成的图像)
|
||||||
|
uploadResults = await uploadImages(imageUrls, accessToken, true);
|
||||||
|
|
||||||
// 检查上传结果
|
// 检查上传结果
|
||||||
const failedUploads = uploadResults.filter(r => !r.success);
|
const failedUploads = uploadResults.filter(r => !r.success);
|
||||||
|
|||||||
@@ -1,5 +1,6 @@
|
|||||||
import { useEffect } from 'react';
|
import { useEffect } from 'react';
|
||||||
import { useAppStore } from '../store/useAppStore';
|
import { useAppStore } from '../store/useAppStore';
|
||||||
|
import { useImageGeneration, useImageEditing } from '../hooks/useImageGeneration';
|
||||||
|
|
||||||
export const useKeyboardShortcuts = () => {
|
export const useKeyboardShortcuts = () => {
|
||||||
const {
|
const {
|
||||||
@@ -9,9 +10,19 @@ export const useKeyboardShortcuts = () => {
|
|||||||
setShowPromptPanel,
|
setShowPromptPanel,
|
||||||
showPromptPanel,
|
showPromptPanel,
|
||||||
currentPrompt,
|
currentPrompt,
|
||||||
isGenerating
|
isGenerating,
|
||||||
|
selectedTool,
|
||||||
|
editReferenceImages,
|
||||||
|
canvasImage,
|
||||||
|
setCanvasImage,
|
||||||
|
temperature,
|
||||||
|
seed,
|
||||||
|
uploadedImages: generateUploadedImages
|
||||||
} = useAppStore();
|
} = useAppStore();
|
||||||
|
|
||||||
|
const { generate } = useImageGeneration();
|
||||||
|
const { edit } = useImageEditing();
|
||||||
|
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
const handleKeyDown = (event: KeyboardEvent) => {
|
const handleKeyDown = (event: KeyboardEvent) => {
|
||||||
// Ignore if user is typing in an input
|
// Ignore if user is typing in an input
|
||||||
@@ -21,7 +32,21 @@ export const useKeyboardShortcuts = () => {
|
|||||||
if ((event.metaKey || event.ctrlKey) && event.key === 'Enter') {
|
if ((event.metaKey || event.ctrlKey) && event.key === 'Enter') {
|
||||||
event.preventDefault();
|
event.preventDefault();
|
||||||
if (!isGenerating && currentPrompt.trim()) {
|
if (!isGenerating && currentPrompt.trim()) {
|
||||||
console.log('Generate via keyboard shortcut');
|
// 触发生成操作
|
||||||
|
if (selectedTool === 'generate') {
|
||||||
|
const referenceImages = generateUploadedImages
|
||||||
|
.filter(img => img.includes('base64,'))
|
||||||
|
.map(img => img.split('base64,')[1]);
|
||||||
|
|
||||||
|
generate({
|
||||||
|
prompt: currentPrompt,
|
||||||
|
referenceImages: referenceImages.length > 0 ? referenceImages : undefined,
|
||||||
|
temperature,
|
||||||
|
seed: seed !== null ? seed : undefined
|
||||||
|
});
|
||||||
|
} else if (selectedTool === 'edit' || selectedTool === 'mask') {
|
||||||
|
edit(currentPrompt);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return;
|
return;
|
||||||
@@ -54,10 +79,47 @@ export const useKeyboardShortcuts = () => {
|
|||||||
console.log('Re-roll variants');
|
console.log('Re-roll variants');
|
||||||
}
|
}
|
||||||
break;
|
break;
|
||||||
|
case 'enter':
|
||||||
|
// 如果按Enter键且有提示词,则触发生成
|
||||||
|
if (currentPrompt.trim() && !isGenerating) {
|
||||||
|
event.preventDefault();
|
||||||
|
if (selectedTool === 'generate') {
|
||||||
|
const referenceImages = generateUploadedImages
|
||||||
|
.filter(img => img.includes('base64,'))
|
||||||
|
.map(img => img.split('base64,')[1]);
|
||||||
|
|
||||||
|
generate({
|
||||||
|
prompt: currentPrompt,
|
||||||
|
referenceImages: referenceImages.length > 0 ? referenceImages : undefined,
|
||||||
|
temperature,
|
||||||
|
seed: seed !== null ? seed : undefined
|
||||||
|
});
|
||||||
|
} else if (selectedTool === 'edit' || selectedTool === 'mask') {
|
||||||
|
edit(currentPrompt);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
break;
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
window.addEventListener('keydown', handleKeyDown);
|
window.addEventListener('keydown', handleKeyDown);
|
||||||
return () => window.removeEventListener('keydown', handleKeyDown);
|
return () => window.removeEventListener('keydown', handleKeyDown);
|
||||||
}, [setSelectedTool, setShowHistory, showHistory, setShowPromptPanel, showPromptPanel, currentPrompt, isGenerating]);
|
}, [
|
||||||
|
setSelectedTool,
|
||||||
|
setShowHistory,
|
||||||
|
showHistory,
|
||||||
|
setShowPromptPanel,
|
||||||
|
showPromptPanel,
|
||||||
|
currentPrompt,
|
||||||
|
isGenerating,
|
||||||
|
selectedTool,
|
||||||
|
generateUploadedImages,
|
||||||
|
editReferenceImages,
|
||||||
|
canvasImage,
|
||||||
|
setCanvasImage,
|
||||||
|
temperature,
|
||||||
|
seed,
|
||||||
|
generate,
|
||||||
|
edit
|
||||||
|
]);
|
||||||
};
|
};
|
||||||
@@ -1,166 +1,168 @@
|
|||||||
import { GoogleGenAI } from '@google/genai';
|
import { GoogleGenAI } from '@google/genai'
|
||||||
|
|
||||||
// 注意:在生产环境中,这应该通过后端代理处理
|
// 注意:在生产环境中,这应该通过后端代理处理
|
||||||
const API_KEY = import.meta.env.VITE_GEMINI_API_KEY || 'demo-key';
|
const API_KEY = import.meta.env.VITE_GEMINI_API_KEY || 'demo-key'
|
||||||
const genAI = new GoogleGenAI({ apiKey: API_KEY });
|
const genAI = new GoogleGenAI({ apiKey: API_KEY })
|
||||||
|
|
||||||
export interface GenerationRequest {
|
export interface GenerationRequest {
|
||||||
prompt: string;
|
prompt: string
|
||||||
referenceImages?: string[]; // base64数组
|
referenceImages?: string[] // base64数组
|
||||||
temperature?: number;
|
temperature?: number
|
||||||
seed?: number;
|
seed?: number
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface EditRequest {
|
export interface EditRequest {
|
||||||
instruction: string;
|
instruction: string
|
||||||
originalImage: string; // base64
|
originalImage: string // base64
|
||||||
referenceImages?: string[]; // base64数组
|
referenceImages?: string[] // base64数组
|
||||||
maskImage?: string; // base64
|
maskImage?: string // base64
|
||||||
temperature?: number;
|
temperature?: number
|
||||||
seed?: number;
|
seed?: number
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface UsageMetadata {
|
export interface UsageMetadata {
|
||||||
totalTokenCount?: number;
|
totalTokenCount?: number
|
||||||
promptTokenCount?: number;
|
promptTokenCount?: number
|
||||||
candidatesTokenCount?: number;
|
candidatesTokenCount?: number
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface SegmentationRequest {
|
export interface SegmentationRequest {
|
||||||
image: string; // base64
|
image: string // base64
|
||||||
query: string; // "像素(x,y)处的对象" 或 "红色汽车"
|
query: string // "像素(x,y)处的对象" 或 "红色汽车"
|
||||||
}
|
}
|
||||||
|
|
||||||
export class GeminiService {
|
export class GeminiService {
|
||||||
async generateImage(request: GenerationRequest): Promise<{images: string[], usageMetadata?: any}> {
|
async generateImage(request: GenerationRequest): Promise<{ images: string[]; usageMetadata?: any }> {
|
||||||
try {
|
try {
|
||||||
const contents: any[] = [{ text: request.prompt }];
|
const contents: any[] = [{ text: request.prompt }]
|
||||||
|
|
||||||
// 如果提供了参考图像则添加
|
// 如果提供了参考图像则添加
|
||||||
if (request.referenceImages && request.referenceImages.length > 0) {
|
if (request.referenceImages && request.referenceImages.length > 0) {
|
||||||
request.referenceImages.forEach(image => {
|
request.referenceImages.forEach(image => {
|
||||||
contents.push({
|
contents.push({
|
||||||
inlineData: {
|
inlineData: {
|
||||||
mimeType: "image/png",
|
mimeType: 'image/png',
|
||||||
data: image,
|
data: image,
|
||||||
},
|
},
|
||||||
});
|
})
|
||||||
});
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
const response = await genAI.models.generateContent({
|
const response = await genAI.models.generateContent({
|
||||||
model: "gemini-2.5-flash-image-preview",
|
model: 'gemini-2.5-flash-image-preview',
|
||||||
contents,
|
contents,
|
||||||
});
|
})
|
||||||
|
|
||||||
// 检查是否有被禁止的内容
|
// 检查是否有被禁止的内容
|
||||||
if (response.candidates && response.candidates.length > 0) {
|
if (response.candidates && response.candidates.length > 0) {
|
||||||
const candidate = response.candidates[0];
|
const candidate = response.candidates[0]
|
||||||
if (candidate.finishReason === 'PROHIBITED_CONTENT') {
|
if (candidate.finishReason === 'PROHIBITED_CONTENT') {
|
||||||
throw new Error('内容被禁止:您的请求包含不允许的内容。请尝试其他提示。');
|
throw new Error('内容被禁止:您的请求包含不允许的内容。请尝试其他提示。')
|
||||||
|
}
|
||||||
|
if (candidate.finishReason === 'IMAGE_SAFETY') {
|
||||||
|
throw new Error('内容被禁止:您的请求包含不允许的内容。请尝试其他提示。')
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
const images: string[] = [];
|
const images: string[] = []
|
||||||
|
|
||||||
// 检查响应是否存在以及是否有内容
|
// 检查响应是否存在以及是否有内容
|
||||||
if (response.candidates && response.candidates.length > 0 &&
|
if (response.candidates && response.candidates.length > 0 && response.candidates[0].content && response.candidates[0].content.parts) {
|
||||||
response.candidates[0].content && response.candidates[0].content.parts) {
|
|
||||||
for (const part of response.candidates[0].content.parts) {
|
for (const part of response.candidates[0].content.parts) {
|
||||||
if (part.inlineData) {
|
if (part.inlineData) {
|
||||||
images.push(part.inlineData.data);
|
images.push(part.inlineData.data)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// 获取usageMetadata(如果存在)
|
// 获取usageMetadata(如果存在)
|
||||||
const usageMetadata = response.usageMetadata;
|
const usageMetadata = response.usageMetadata
|
||||||
|
|
||||||
return { images, usageMetadata };
|
return { images, usageMetadata }
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
console.error('生成图像时出错:', error);
|
console.error('生成图像时出错:', error)
|
||||||
if (error instanceof Error && error.message) {
|
if (error instanceof Error && error.message) {
|
||||||
throw error;
|
throw error
|
||||||
}
|
}
|
||||||
throw new Error(`生成图像失败: ${error instanceof Error ? error.message : '未知错误'}`);
|
throw new Error(`生成图像失败: ${error instanceof Error ? error.message : '未知错误'}`)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
async editImage(request: EditRequest): Promise<{images: string[], usageMetadata?: any}> {
|
async editImage(request: EditRequest): Promise<{ images: string[]; usageMetadata?: any }> {
|
||||||
try {
|
try {
|
||||||
const contents = [
|
const contents = [
|
||||||
{ text: this.buildEditPrompt(request) },
|
{ text: this.buildEditPrompt(request) },
|
||||||
{
|
{
|
||||||
inlineData: {
|
inlineData: {
|
||||||
mimeType: "image/png",
|
mimeType: 'image/png',
|
||||||
data: request.originalImage,
|
data: request.originalImage,
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
];
|
]
|
||||||
|
|
||||||
// 如果提供了参考图像则添加
|
// 如果提供了参考图像则添加
|
||||||
if (request.referenceImages && request.referenceImages.length > 0) {
|
if (request.referenceImages && request.referenceImages.length > 0) {
|
||||||
request.referenceImages.forEach(image => {
|
request.referenceImages.forEach(image => {
|
||||||
contents.push({
|
contents.push({
|
||||||
inlineData: {
|
inlineData: {
|
||||||
mimeType: "image/png",
|
mimeType: 'image/png',
|
||||||
data: image,
|
data: image,
|
||||||
},
|
},
|
||||||
});
|
})
|
||||||
});
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
if (request.maskImage) {
|
if (request.maskImage) {
|
||||||
contents.push({
|
contents.push({
|
||||||
inlineData: {
|
inlineData: {
|
||||||
mimeType: "image/png",
|
mimeType: 'image/png',
|
||||||
data: request.maskImage,
|
data: request.maskImage,
|
||||||
},
|
},
|
||||||
});
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
const response = await genAI.models.generateContent({
|
const response = await genAI.models.generateContent({
|
||||||
model: "gemini-2.5-flash-image-preview",
|
model: 'gemini-2.5-flash-image-preview',
|
||||||
contents,
|
contents,
|
||||||
});
|
})
|
||||||
|
|
||||||
// 检查是否有被禁止的内容
|
// 检查是否有被禁止的内容
|
||||||
if (response.candidates && response.candidates.length > 0) {
|
if (response.candidates && response.candidates.length > 0) {
|
||||||
const candidate = response.candidates[0];
|
const candidate = response.candidates[0]
|
||||||
if (candidate.finishReason === 'PROHIBITED_CONTENT') {
|
if (candidate.finishReason === 'PROHIBITED_CONTENT') {
|
||||||
throw new Error('内容被禁止:您的请求包含不允许的内容。请尝试其他提示。');
|
throw new Error('内容被禁止:您的请求包含不允许的内容。请尝试其他提示。')
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
const images: string[] = [];
|
const images: string[] = []
|
||||||
|
|
||||||
// 检查响应是否存在以及是否有内容
|
// 检查响应是否存在以及是否有内容
|
||||||
if (response.candidates && response.candidates.length > 0 &&
|
if (response.candidates && response.candidates.length > 0 && response.candidates[0].content && response.candidates[0].content.parts) {
|
||||||
response.candidates[0].content && response.candidates[0].content.parts) {
|
|
||||||
for (const part of response.candidates[0].content.parts) {
|
for (const part of response.candidates[0].content.parts) {
|
||||||
if (part.inlineData) {
|
if (part.inlineData) {
|
||||||
images.push(part.inlineData.data);
|
images.push(part.inlineData.data)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// 获取usageMetadata(如果存在)
|
// 获取usageMetadata(如果存在)
|
||||||
const usageMetadata = response.usageMetadata;
|
const usageMetadata = response.usageMetadata
|
||||||
|
|
||||||
return { images, usageMetadata };
|
return { images, usageMetadata }
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
console.error('编辑图像时出错:', error);
|
console.error('编辑图像时出错:', error)
|
||||||
if (error instanceof Error && error.message) {
|
if (error instanceof Error && error.message) {
|
||||||
throw error;
|
throw error
|
||||||
}
|
}
|
||||||
throw new Error(`编辑图像失败: ${error instanceof Error ? error.message : '未知错误'}`);
|
throw new Error(`编辑图像失败: ${error instanceof Error ? error.message : '未知错误'}`)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
async segmentImage(request: SegmentationRequest): Promise<any> {
|
async segmentImage(request: SegmentationRequest): Promise<any> {
|
||||||
try {
|
try {
|
||||||
const prompt = [
|
const prompt = [
|
||||||
{ text: `分析此图像并为以下对象创建分割遮罩: ${request.query}
|
{
|
||||||
|
text: `分析此图像并为以下对象创建分割遮罩: ${request.query}
|
||||||
|
|
||||||
返回具有此确切结构的JSON对象:
|
返回具有此确切结构的JSON对象:
|
||||||
{
|
{
|
||||||
@@ -173,50 +175,49 @@ export class GeminiService {
|
|||||||
]
|
]
|
||||||
}
|
}
|
||||||
|
|
||||||
仅分割请求的特定对象或区域。遮罩应该是二进制PNG,其中白色像素(255)表示选定区域,黑色像素(0)表示背景。` },
|
仅分割请求的特定对象或区域。遮罩应该是二进制PNG,其中白色像素(255)表示选定区域,黑色像素(0)表示背景。`,
|
||||||
|
},
|
||||||
{
|
{
|
||||||
inlineData: {
|
inlineData: {
|
||||||
mimeType: "image/png",
|
mimeType: 'image/png',
|
||||||
data: request.image,
|
data: request.image,
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
];
|
]
|
||||||
|
|
||||||
const response = await genAI.models.generateContent({
|
const response = await genAI.models.generateContent({
|
||||||
model: "gemini-2.5-flash-image-preview",
|
model: 'gemini-2.5-flash-image-preview',
|
||||||
contents: prompt,
|
contents: prompt,
|
||||||
});
|
})
|
||||||
|
|
||||||
// 检查是否有被禁止的内容
|
// 检查是否有被禁止的内容
|
||||||
if (response.candidates && response.candidates.length > 0) {
|
if (response.candidates && response.candidates.length > 0) {
|
||||||
const candidate = response.candidates[0];
|
const candidate = response.candidates[0]
|
||||||
if (candidate.finishReason === 'PROHIBITED_CONTENT') {
|
if (candidate.finishReason === 'PROHIBITED_CONTENT') {
|
||||||
throw new Error('内容被禁止:您的请求包含不允许的内容。请尝试其他提示。');
|
throw new Error('内容被禁止:您的请求包含不允许的内容。请尝试其他提示。')
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
const responseText = response.candidates[0].content.parts[0].text;
|
const responseText = response.candidates[0].content.parts[0].text
|
||||||
return JSON.parse(responseText);
|
return JSON.parse(responseText)
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
console.error('分割图像时出错:', error);
|
console.error('分割图像时出错:', error)
|
||||||
if (error instanceof Error && error.message) {
|
if (error instanceof Error && error.message) {
|
||||||
throw error;
|
throw error
|
||||||
}
|
}
|
||||||
throw new Error(`分割图像失败: ${error instanceof Error ? error.message : '未知错误'}`);
|
throw new Error(`分割图像失败: ${error instanceof Error ? error.message : '未知错误'}`)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private buildEditPrompt(request: EditRequest): string {
|
private buildEditPrompt(request: EditRequest): string {
|
||||||
const maskInstruction = request.maskImage
|
const maskInstruction = request.maskImage ? '\n\n重要: 仅在遮罩图像显示白色像素(值255)的地方应用更改。完全不更改所有其他区域。精确遵守遮罩边界并在边缘保持无缝混合。' : ''
|
||||||
? "\n\n重要: 仅在遮罩图像显示白色像素(值255)的地方应用更改。完全不更改所有其他区域。精确遵守遮罩边界并在边缘保持无缝混合。"
|
|
||||||
: "";
|
|
||||||
|
|
||||||
return `根据以下指令编辑此图像: ${request.instruction}
|
return `根据以下指令编辑此图像: ${request.instruction}
|
||||||
|
|
||||||
保持原始图像的光照、透视和整体构图。使更改看起来自然且无缝集成。${maskInstruction}
|
保持原始图像的光照、透视和整体构图。使更改看起来自然且无缝集成。${maskInstruction}
|
||||||
|
|
||||||
保持图像质量并确保编辑看起来专业且逼真。`;
|
保持图像质量并确保编辑看起来专业且逼真。`
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
export const geminiService = new GeminiService();
|
export const geminiService = new GeminiService()
|
||||||
|
|||||||
@@ -4,13 +4,41 @@ import { UploadResult } from '../types'
|
|||||||
// 上传接口URL
|
// 上传接口URL
|
||||||
const UPLOAD_URL = 'https://api.pandorastudio.cn/auth/OSSupload'
|
const UPLOAD_URL = 'https://api.pandorastudio.cn/auth/OSSupload'
|
||||||
|
|
||||||
|
// 创建一个Map来缓存已上传的图像
|
||||||
|
const uploadCache = new Map<string, UploadResult>()
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 生成图像的唯一标识符
|
||||||
|
* @param base64Data - base64编码的图像数据
|
||||||
|
* @returns 图像的唯一标识符
|
||||||
|
*/
|
||||||
|
function getImageHash(base64Data: string): string {
|
||||||
|
// 使用简单的哈希函数生成图像标识符
|
||||||
|
let hash = 0;
|
||||||
|
for (let i = 0; i < base64Data.length; i++) {
|
||||||
|
const char = base64Data.charCodeAt(i);
|
||||||
|
hash = ((hash << 5) - hash) + char;
|
||||||
|
hash = hash & hash; // 转换为32位整数
|
||||||
|
}
|
||||||
|
return hash.toString();
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* 将base64图像数据上传到指定接口
|
* 将base64图像数据上传到指定接口
|
||||||
* @param base64Data - base64编码的图像数据
|
* @param base64Data - base64编码的图像数据
|
||||||
* @param accessToken - 访问令牌
|
* @param accessToken - 访问令牌
|
||||||
|
* @param skipCache - 是否跳过缓存检查
|
||||||
* @returns 上传结果
|
* @returns 上传结果
|
||||||
*/
|
*/
|
||||||
export const uploadImage = async (base64Data: string, accessToken: string): Promise<{ success: boolean; url?: string; error?: string }> => {
|
export const uploadImage = async (base64Data: string, accessToken: string, skipCache: boolean = false): Promise<{ success: boolean; url?: string; error?: string }> => {
|
||||||
|
// 检查缓存中是否已有该图像的上传结果
|
||||||
|
const imageHash = getImageHash(base64Data)
|
||||||
|
|
||||||
|
if (!skipCache && uploadCache.has(imageHash)) {
|
||||||
|
console.log('从缓存中获取上传结果')
|
||||||
|
return uploadCache.get(imageHash)!
|
||||||
|
}
|
||||||
|
|
||||||
try {
|
try {
|
||||||
// 将base64数据转换为Blob
|
// 将base64数据转换为Blob
|
||||||
const byteString = atob(base64Data.split(',')[1])
|
const byteString = atob(base64Data.split(',')[1])
|
||||||
@@ -44,13 +72,29 @@ export const uploadImage = async (base64Data: string, accessToken: string): Prom
|
|||||||
// 使用环境变量中的VITE_UPLOAD_ASSET_URL作为前缀
|
// 使用环境变量中的VITE_UPLOAD_ASSET_URL作为前缀
|
||||||
const uploadAssetUrl = import.meta.env.VITE_UPLOAD_ASSET_URL || ''
|
const uploadAssetUrl = import.meta.env.VITE_UPLOAD_ASSET_URL || ''
|
||||||
const fullUrl = uploadAssetUrl ? `${uploadAssetUrl}/${result.data}` : result.data
|
const fullUrl = uploadAssetUrl ? `${uploadAssetUrl}/${result.data}` : result.data
|
||||||
return { success: true, url: fullUrl, error: undefined }
|
|
||||||
|
// 将上传结果存储到缓存中
|
||||||
|
const uploadResult = { success: true, url: fullUrl, error: undefined }
|
||||||
|
uploadCache.set(imageHash, {
|
||||||
|
...uploadResult,
|
||||||
|
timestamp: Date.now()
|
||||||
|
})
|
||||||
|
|
||||||
|
return uploadResult
|
||||||
} else {
|
} else {
|
||||||
throw new Error(`上传失败: ${result.msg}`)
|
throw new Error(`上传失败: ${result.msg}`)
|
||||||
}
|
}
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
console.error('上传图像时出错:', error)
|
console.error('上传图像时出错:', error)
|
||||||
return { success: false, url: undefined, error: error instanceof Error ? error.message : String(error) }
|
const errorResult = { success: false, url: undefined, error: error instanceof Error ? error.message : String(error) }
|
||||||
|
|
||||||
|
// 将失败的上传结果也存储到缓存中(可选)
|
||||||
|
uploadCache.set(imageHash, {
|
||||||
|
...errorResult,
|
||||||
|
timestamp: Date.now()
|
||||||
|
})
|
||||||
|
|
||||||
|
return errorResult
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -58,16 +102,17 @@ export const uploadImage = async (base64Data: string, accessToken: string): Prom
|
|||||||
* 上传多个图像
|
* 上传多个图像
|
||||||
* @param base64Images - base64编码的图像数组
|
* @param base64Images - base64编码的图像数组
|
||||||
* @param accessToken - 访问令牌
|
* @param accessToken - 访问令牌
|
||||||
|
* @param skipCache - 是否跳过缓存检查
|
||||||
* @returns 上传结果数组
|
* @returns 上传结果数组
|
||||||
*/
|
*/
|
||||||
export const uploadImages = async (base64Images: string[], accessToken: string): Promise<UploadResult[]> => {
|
export const uploadImages = async (base64Images: string[], accessToken: string, skipCache: boolean = false): Promise<UploadResult[]> => {
|
||||||
try {
|
try {
|
||||||
const results: UploadResult[] = []
|
const results: UploadResult[] = []
|
||||||
|
|
||||||
for (let i = 0; i < base64Images.length; i++) {
|
for (let i = 0; i < base64Images.length; i++) {
|
||||||
const base64Data = base64Images[i]
|
const base64Data = base64Images[i]
|
||||||
try {
|
try {
|
||||||
const uploadResult = await uploadImage(base64Data, accessToken)
|
const uploadResult = await uploadImage(base64Data, accessToken, skipCache)
|
||||||
const result: UploadResult = {
|
const result: UploadResult = {
|
||||||
success: uploadResult.success,
|
success: uploadResult.success,
|
||||||
url: uploadResult.url,
|
url: uploadResult.url,
|
||||||
@@ -101,3 +146,10 @@ export const uploadImages = async (base64Images: string[], accessToken: string):
|
|||||||
throw error
|
throw error
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 清除上传缓存
|
||||||
|
*/
|
||||||
|
export const clearUploadCache = (): void => {
|
||||||
|
uploadCache.clear()
|
||||||
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user