一、CLIP-I、CLIP-T、DINO指标的介绍

CLIP论文的介绍
Learning Transferable Visual Models From Natural Language Supervision

以下是对 CLIP-I、CLIP-T 和 DINO 这三个指标的清晰介绍，涵盖它们的计算方式、用途以及适用场景：

---

🔶 1. CLIP-I（CLIP Image-to-Image Similarity）

✅ 简介：

CLIP-I 衡量的是两张图像之间的视觉语义相似度，是通过 CLIP 模型 的 图像编码器 提取特征后进行余弦相似度计算。

✅ 计算方法：

• 用 CLIP 的图像编码器提取两张图像的特征向量。
• 对两个向量进行归一化。
• 使用 cosine similarity 计算相似度。

similarity = cosine_similarity(image_feat1, image_feat2)

✅ 作用：

• 判断两张图像是否在高层语义上相似。
• 常用于图像风格迁移、图像重建等视觉质量评价任务。

✅ 特点：

• 比像素级更具语义性。
• 与人类感知更一致。
• 不涉及文本，适合图像对比。

---

🔷 2. CLIP-T（CLIP Image-to-Text Similarity）

✅ 简介：

CLIP-T 衡量的是一张图像与一个文本描述之间的语义匹配度，是 CLIP 最核心的用途。

✅ 计算方法：

• 用 CLIP 的图像编码器提取图像特征。
• 用 CLIP 的文本编码器提取文本特征。
• 两者归一化后计算 cosine 相似度。

similarity = cosine_similarity(image_feat, text_feat)

✅ 作用：

• 评估生成图像是否准确反映文本内容（文本图像一致性）。
• 常用于文图检索、图像生成评估（如 DALL·E、Stable Diffusion）。

✅ 特点：

• 直接衡量图文语义是否对应。
• 是 prompt-driven 图像生成评估的关键指标之一。

---

🔸 3. DINO（Self-Supervised Vision Similarity）

✅ 简介：

DINO 是一种无监督学习的图像表示模型（由 Meta 提出），基于 Vision Transformer（ViT）。它没有使用文字标签，仅通过对比学习学习图像的结构化语义表示。

✅ 计算方法：

• 使用 DINO 模型提取图像的最后一层特征。
• 取平均（global average pooling）形成图像向量。
• 计算两张图像的余弦相似度。

similarity = cosine_similarity(dino_feat1, dino_feat2)

✅ 作用：

• 用于图像-图像的相似性评估。
• 更注重图像的视觉结构、纹理和高层次语义。

✅ 特点：

• 不依赖任何标签或文本信息。
• 在图像对比、迁移、重建质量评估上表现强。
• 与 CLIP-I 相似，但训练方式不同。

---

🔍 总结对比

指标	适用对象	依赖文本	特点	常用任务
CLIP-I	图像 vs 图像	否	语义相似，感知一致性好	图像重建、图像检索、风格迁移
CLIP-T	图像 vs 文本	✅	图文对齐，评估文本忠实性	文图生成、prompt 对应评估
DINO	图像 vs 图像	否	自监督表示，关注结构和形状	表示学习、视觉质量、聚类、对比

---

如果你正在做图像生成模型的定量评估，你通常会：

• 用 CLIP-T 评估生成图像是否“符合 prompt”。
• 用 CLIP-I 和 DINO 评估图像是否接近目标图像或保留结构信息。

二、代码

参考的论文计算

CLIP

参考的方法

2.1CLIP

2.1.1部署

安装包

pip install clip-score

然后下载预预训练权重：clip-vit-base-patch32
然后将要计算的图像和文本分别放在一个文件夹里，注意命名规范（名字和图片需要一一对应），如下

整个clip_score.py的代码如下

"""Calculates the CLIP Scores

The CLIP model is a contrasitively learned language-image model. There is
an image encoder and a text encoder. It is believed that the CLIP model could
measure the similarity of cross modalities. Please find more information from
https://github.com/openai/CLIP.

The CLIP Score measures the Cosine Similarity between two embedded features.
This repository utilizes the pretrained CLIP Model to calculate
the mean average of cosine similarities.

See --help to see further details.

Code adapted from https://github.com/mseitzer/pytorch-fid and
https://github.com/openai/CLIP.

Copyright 2025 The Chinese University of Hong Kong, Shenzhen

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

   http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
"""
import os
import os.path as osp
from argparse import ArgumentDefaultsHelpFormatter, ArgumentParser

import torch
from PIL import Image
from torch.utils.data import DataLoader, Dataset
from tqdm import tqdm
from transformers import AutoModel, AutoProcessor, AutoTokenizer

parser = ArgumentParser(formatter_class=ArgumentDefaultsHelpFormatter)
parser.add_argument('--batch-size',
                    type=int,
                    default=50,
                    help='Batch size to use')
parser.add_argument('--clip-model',
                    type=str,
                    default='/media/dell/DATA/RK/pretrained_model/clip-vit-base-patch32',
                    help='CLIP model to use')
parser.add_argument('--num-workers',
                    type=int,
                    help=('Number of processes to use for data loading. '
                          'Defaults to `min(8, num_cpus)`'))
parser.add_argument('--device',
                    type=str,
                    default=None,
                    help='Device to use. Like cuda, cuda:0 or cpu')
parser.add_argument('--real_flag',
                    type=str,
                    default='img',
                    help=('The modality of real path. '
                          'Default to img'))
parser.add_argument('--fake_flag',
                    type=str,
                    default='txt',
                    help=('The modality of real path. '
                          'Default to txt'))
parser.add_argument('real_path', type=str, help=('Path to your modality data'))
parser.add_argument('fake_path', type=str, help=('Path to your modality data'))

IMAGE_EXTENSIONS = {
    'bmp', 'jpg', 'jpeg', 'pgm', 'png', 'ppm', 'tif', 'tiff', 'webp'
}

TEXT_EXTENSIONS = {'txt'}


class DummyDataset(Dataset):

    FLAGS = ['img', 'txt']

    def __init__(self,
                 real_path,
                 fake_path,
                 real_flag: str = 'img',
                 fake_flag: str = 'txt',
                 transform=None,
                 tokenizer=None) -> None:
        super().__init__()
        if real_flag not in self.FLAGS or fake_flag not in self.FLAGS:
            raise TypeError('CLIP Score only support modality of {}. '
                            'However, get {} and {}'.format(
                                self.FLAGS, real_flag, fake_flag))
        self.real_folder = self._combine_without_prefix(real_path)
        self.real_flag = real_flag
        self.fake_folder = self._combine_without_prefix(fake_path)
        self.fake_flag = fake_flag
        self.transform = transform
        self.tokenizer = tokenizer
        # assert self._check()

    def __len__(self):
        if isinstance(self.real_folder, list):
            real_folder_lenghth = len(self.real_folder)
        else:
            real_folder_lenghth = 1
        if isinstance(self.fake_folder, list):
            fake_folder_lenghth = len(self.fake_folder)
        else:
            fake_folder_lenghth = 1
        return max(real_folder_lenghth, fake_folder_lenghth)

    def __getitem__(self, index):
        if index >= len(self):
            raise IndexError
        if isinstance(self.real_folder, list):
            real_path = self.real_folder[index]
        else:
            real_path = self.real_folder
        if isinstance(self.fake_folder, list):
            fake_path = self.fake_folder[index]
        else:
            fake_path = self.fake_folder
        real_data = self._load_modality(real_path, self.real_flag)
        fake_data = self._load_modality(fake_path, self.fake_flag)

        sample = dict(real=real_data, fake=fake_data)
        return sample

    def _load_modality(self, path, modality):
        if modality == 'img':
            data = self._load_img(path)
        elif modality == 'txt':
            data = self._load_txt(path)
        else:
            raise TypeError('Got unexpected modality: {}'.format(modality))
        return data

    def _load_img(self, path):
        img = Image.open(path)
        if self.transform is not None:
            img = self.transform(text=None, images=img)
            img['pixel_values'] = img['pixel_values'][0]
        return img

    def _load_txt(self, path):
        if osp.exists(path):
            with open(path, 'r') as fp:
                data = fp.read()
                fp.close()
        else:
            data = path
        if self.transform is not None:
            data = self.tokenizer(data, padding=True, return_tensors='pt')
            for key in data:
                data[key] = data[key].squeeze()
        return data

    def _check(self):
        for idx in range(len(self)):
            real_name = self.real_folder[idx].split('.')
            fake_name = self.fake_folder[idx].split('.')
            if fake_name != real_name:
                return False
        return True

    def _combine_without_prefix(self, folder_path, prefix='.'):
        if not osp.exists(folder_path):
            return folder_path
        folder = []
        for name in os.listdir(folder_path):
            if name[0] == prefix:
                continue
            folder.append(osp.join(folder_path, name))
        folder.sort()
        return folder


@torch.no_grad()
def calculate_clip_score(dataloader, model, real_flag, fake_flag):
    score_acc = 0.
    sample_num = 0.
    for batch_data in tqdm(dataloader):
        real = batch_data['real']
        real_features = forward_modality(model, real, real_flag)
        fake = batch_data['fake']
        fake_features = forward_modality(model, fake, fake_flag)

        # normalize features
        real_features = real_features / real_features.norm(
            dim=1, keepdim=True).to(torch.float32)
        fake_features = fake_features / fake_features.norm(
            dim=1, keepdim=True).to(torch.float32)

        # calculate scores
        score = (fake_features * real_features).sum()
        score_acc += score
        sample_num += real_features.shape[0]

    return score_acc / sample_num


def forward_modality(model, data, flag):
    device = next(model.parameters()).device
    for key in data:
        data[key] = data[key].to(device)
    if flag == 'img':
        features = model.get_image_features(**data)
    elif flag == 'txt':
        features = model.get_text_features(**data)
    else:
        raise TypeError
    return features


def main():
    args = parser.parse_args()

    if args.device is None:
        device = torch.device('cuda' if (torch.cuda.is_available()) else 'cpu')
    else:
        device = torch.device(args.device)

    if args.num_workers is None:
        try:
            num_cpus = len(os.sched_getaffinity(0))
        except AttributeError:
            # os.sched_getaffinity is not available under Windows, use
            # os.cpu_count instead (which may not return the *available* number
            # of CPUs).
            num_cpus = os.cpu_count()

        num_workers = min(num_cpus, 8) if num_cpus is not None else 0
    else:
        num_workers = args.num_workers

    print('Loading CLIP model: {}'.format(args.clip_model))
    model = AutoModel.from_pretrained(args.clip_model).to(device)
    processor = AutoProcessor.from_pretrained(args.clip_model)
    tokenizer = AutoTokenizer.from_pretrained(args.clip_model)

    dataset = DummyDataset(args.real_path,
                           args.fake_path,
                           args.real_flag,
                           args.fake_flag,
                           transform=processor,
                           tokenizer=tokenizer)
    dataloader = DataLoader(dataset,
                            args.batch_size,
                            num_workers=num_workers,
                            pin_memory=True)

    print('Calculating CLIP Score:')
    clip_score = calculate_clip_score(dataloader, model, args.real_flag,
                                      args.fake_flag)
    clip_score = clip_score.cpu().item()
    print('CLIP Score: ', clip_score)


if __name__ == '__main__':
    main()

2.1.2CLIP-I

python -m clip_score '//media/dell/DATA/RK/StyleCrafter-main/output/style_image_generation/samples' '/media/dell/DATA/RK/StyleCrafter-main/output/style_image_generation/style' --real_flag img --fake_flag img

多个文本会报错，只能一个文本

python -m clip_score '/media/dell/DATA/RK/StyleCrafter-main/cal_CLIP/te_data/iamge' 'dog'

需要修改一下代码才能正确使用

"""Calculates the CLIP Scores

The CLIP model is a contrasitively learned language-image model. There is
an image encoder and a text encoder. It is believed that the CLIP model could
measure the similarity of cross modalities. Please find more information from
https://github.com/openai/CLIP.

The CLIP Score measures the Cosine Similarity between two embedded features.
This repository utilizes the pretrained CLIP Model to calculate
the mean average of cosine similarities.

See --help to see further details.

Code adapted from https://github.com/mseitzer/pytorch-fid and
https://github.com/openai/CLIP.

Copyright 2025 The Chinese University of Hong Kong, Shenzhen

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

   http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
"""
import os
import os.path as osp
from argparse import ArgumentDefaultsHelpFormatter, ArgumentParser

import torch
from PIL import Image
from torch.utils.data import DataLoader, Dataset
from tqdm import tqdm
from transformers import AutoModel, AutoProcessor, AutoTokenizer

parser = ArgumentParser(formatter_class=ArgumentDefaultsHelpFormatter)
parser.add_argument('--batch-size',
                    type=int,
                    default=50,
                    help='Batch size to use')
parser.add_argument('--clip-model',
                    type=str,
                    default='/media/dell/DATA/RK/pretrained_model/clip-vit-base-patch32',
                    help='CLIP model to use')
parser.add_argument('--num-workers',
                    type=int,
                    help=('Number of processes to use for data loading. '
                          'Defaults to `min(8, num_cpus)`'))
parser.add_argument('--device',
                    type=str,
                    default=None,
                    help='Device to use. Like cuda, cuda:0 or cpu')
parser.add_argument('--real_flag',
                    type=str,
                    default='img',
                    help=('The modality of real path. '
                          'Default to img'))
parser.add_argument('--fake_flag',
                    type=str,
                    default='txt',
                    help=('The modality of real path. '
                          'Default to txt'))
parser.add_argument('real_path', type=str, help=('Path to your modality data'))
parser.add_argument('fake_path', type=str, help=('Path to your modality data'))

IMAGE_EXTENSIONS = {
    'bmp', 'jpg', 'jpeg', 'pgm', 'png', 'ppm', 'tif', 'tiff', 'webp'
}

TEXT_EXTENSIONS = {'txt'}


class DummyDataset(Dataset):

    FLAGS = ['img', 'txt']

    def __init__(self,
                 real_path,
                 fake_path,
                 real_flag: str = 'img',
                 fake_flag: str = 'txt',
                 transform=None,
                 tokenizer=None) -> None:
        super().__init__()
        if real_flag not in self.FLAGS or fake_flag not in self.FLAGS:
            raise TypeError('CLIP Score only support modality of {}. '
                            'However, get {} and {}'.format(
                                self.FLAGS, real_flag, fake_flag))
        self.real_folder = self._combine_without_prefix(real_path)
        self.real_flag = real_flag
        self.fake_folder = self._combine_without_prefix(fake_path)
        self.fake_flag = fake_flag
        self.transform = transform
        self.tokenizer = tokenizer
        # assert self._check()

    def __len__(self):
        if isinstance(self.real_folder, list):
            real_folder_lenghth = len(self.real_folder)
        else:
            real_folder_lenghth = 1
        if isinstance(self.fake_folder, list):
            fake_folder_lenghth = len(self.fake_folder)
        else:
            fake_folder_lenghth = 1
        return max(real_folder_lenghth, fake_folder_lenghth)

    def __getitem__(self, index):
        if index >= len(self):
            raise IndexError
        if isinstance(self.real_folder, list):
            real_path = self.real_folder[index]
        else:
            real_path = self.real_folder
        if isinstance(self.fake_folder, list):
            fake_path = self.fake_folder[index]
        else:
            fake_path = self.fake_folder
        real_data = self._load_modality(real_path, self.real_flag)
        fake_data = self._load_modality(fake_path, self.fake_flag)

        sample = dict(real=real_data, fake=fake_data)
        return sample

    def _load_modality(self, path, modality):
        if modality == 'img':
            data = self._load_img(path)
        elif modality == 'txt':
            data = self._load_txt(path)
        else:
            raise TypeError('Got unexpected modality: {}'.format(modality))
        return data

    def _load_img(self, path):
        img = Image.open(path)
        if self.transform is not None:
            img = self.transform(text=None, images=img)
            img['pixel_values'] = img['pixel_values'][0]
        return img

    
    def _load_txt(self, path):
        if osp.exists(path):
            with open(path, 'r') as fp:
                data = fp.read()
                fp.close()
        else:
            data = path
        if self.transform is not None:
            #data = self.tokenizer(data, padding=True, return_tensors='pt')
            #for key in data:
            #    data[key] = data[key].squeeze()
            data = self.tokenizer(data, padding='max_length', truncation=True, return_tensors='pt')
            for key in data:
                data[key] = data[key][0]  # 取第一个样本而不是 squeeze 全部维度

        return data

    def _check(self):
        for idx in range(len(self)):
            real_name = self.real_folder[idx].split('.')
            fake_name = self.fake_folder[idx].split('.')
            if fake_name != real_name:
                return False
        return True

    def _combine_without_prefix(self, folder_path, prefix='.'):
        if not osp.exists(folder_path):
            return folder_path
        folder = []
        for name in os.listdir(folder_path):
            if name[0] == prefix:
                continue
            folder.append(osp.join(folder_path, name))
        folder.sort()
        return folder


@torch.no_grad()
def calculate_clip_score(dataloader, model, real_flag, fake_flag):
    score_acc = 0.
    sample_num = 0.
    for batch_data in tqdm(dataloader):
        real = batch_data['real']
        real_features = forward_modality(model, real, real_flag)
        fake = batch_data['fake']
        fake_features = forward_modality(model, fake, fake_flag)

        # normalize features
        real_features = real_features / real_features.norm(
            dim=1, keepdim=True).to(torch.float32)
        fake_features = fake_features / fake_features.norm(
            dim=1, keepdim=True).to(torch.float32)

        # calculate scores
        score = (fake_features * real_features).sum()
        score_acc += score
        sample_num += real_features.shape[0]

    return score_acc / sample_num


def forward_modality(model, data, flag):
    device = next(model.parameters()).device
    for key in data:
        data[key] = data[key].to(device)
    if flag == 'img':
        features = model.get_image_features(**data)
    elif flag == 'txt':
        features = model.get_text_features(**data)

    else:
        raise TypeError
    return features


def main():
    args = parser.parse_args()

    if args.device is None:
        device = torch.device('cuda' if (torch.cuda.is_available()) else 'cpu')
    else:
        device = torch.device(args.device)

    if args.num_workers is None:
        try:
            num_cpus = len(os.sched_getaffinity(0))
        except AttributeError:
            # os.sched_getaffinity is not available under Windows, use
            # os.cpu_count instead (which may not return the *available* number
            # of CPUs).
            num_cpus = os.cpu_count()

        num_workers = min(num_cpus, 8) if num_cpus is not None else 0
    else:
        num_workers = args.num_workers

    print('Loading CLIP model: {}'.format(args.clip_model))
    model = AutoModel.from_pretrained(args.clip_model).to(device)
    processor = AutoProcessor.from_pretrained(args.clip_model)
    tokenizer = AutoTokenizer.from_pretrained(args.clip_model)

    dataset = DummyDataset(args.real_path,
                           args.fake_path,
                           args.real_flag,
                           args.fake_flag,
                           transform=processor,
                           tokenizer=tokenizer)
    dataloader = DataLoader(dataset,
                            args.batch_size,
                            num_workers=num_workers,
                            pin_memory=True)

    print('Calculating CLIP Score:')
    clip_score = calculate_clip_score(dataloader, model, args.real_flag,
                                      args.fake_flag)
    clip_score = clip_score.cpu().item()
    print('CLIP Score: ', clip_score)


if __name__ == '__main__':
    main()

把def _load_txt(self, path)这个函数进行了修改就可以了

 def _load_txt(self, path):
        if osp.exists(path):
            with open(path, 'r') as fp:
                data = fp.read()
                fp.close()
        else:
            data = path
        if self.transform is not None:
            #data = self.tokenizer(data, padding=True, return_tensors='pt')
            #for key in data:
            #    data[key] = data[key].squeeze()
            data = self.tokenizer(data, padding='max_length', truncation=True, return_tensors='pt')
            for key in data:
                data[key] = data[key][0]  # 取第一个样本而不是 squeeze 全部维度

2.1.3CLIP-T

 python -m clip_score '/media/dell/DATA/RK/StyleCrafter-main/cal_CLIP/te_data/iamge' '/media/dell/DATA/RK/StyleCrafter-main/cal_CLIP/te_data/text' --real_flag img --fake_flag txt

2.1DINO

from transformers import AutoImageProcessor, AutoModel
from PIL import Image
import torch.nn as nn
import torch
import os

# 定义环境
device = torch.device('cuda' if torch.cuda.is_available() else "cpu")

# 加载 dinov2 模型
model_folder = '/media/dell/DATA/RK/pretrained_model//dinov2-base'
processor = AutoImageProcessor.from_pretrained(model_folder)
model = AutoModel.from_pretrained(model_folder).to(device)

# 加载并处理真实图像
real_image = Image.open('/media/dell/DATA/RK/StyleCrafter-main/output/style_image_generation/samples/0000_0.jpg')
with torch.no_grad():
    inputs1 = processor(images=real_image, return_tensors="pt").to(device)
    outputs1 = model(**inputs1)
    image_features1 = outputs1.last_hidden_state
    image_features1 = image_features1.mean(dim=1)

# 初始化相似度计算
cos = nn.CosineSimilarity(dim=0)

# 设置生成图像文件夹路径
generated_image_folder = '/media/dell/DATA/RK/StyleCrafter-main/output/style_image_generation/samples'

# 初始化相似度总和和计数器
total_similarity = 0
count = 0

# 遍历生成图像文件夹并计算相似度
for filename in os.listdir(generated_image_folder):
    if filename.endswith(".png") or filename.endswith(".jpg"):
        image_path = os.path.join(generated_image_folder, filename)

        # 加载生成图像并处理
        generated_image = Image.open(image_path)
        with torch.no_grad():
            inputs2 = processor(images=generated_image, return_tensors="pt").to(device)
            outputs2 = model(**inputs2)
            image_features2 = outputs2.last_hidden_state
            image_features2 = image_features2.mean(dim=1)

        # 计算相似度
        sim = cos(image_features1[0], image_features2[0]).item()
        sim = (sim + 1) / 2  # 将相似度值归一化到 [0, 1] 范围

        # 累加相似度值和计数
        total_similarity += sim
        count += 1
        print(f'real_image和{filename}的相似度值: {sim}')

# 计算平均相似度并输出
if count > 0:
    average_similarity = total_similarity / count
    print(f'DINO score: {average_similarity}')
else:
    print('没有找到生成图像。')

三、测试加速计算

3.1 CLIP-I

import os
from collections import defaultdict

from transformers import AutoModel, AutoProcessor
from PIL import Image
import torch

def load_clip_model(clip_model_path):
    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
    model = AutoModel.from_pretrained(clip_model_path).to(device)
    processor = AutoProcessor.from_pretrained(clip_model_path)
    return model, processor, device

def compute_batch_clip_similarities(image_paths, target_image_path, model, processor, device):
    # 加载并预处理目标图像
    target_image = Image.open(target_image_path).convert("RGB")
    target_input = processor(images=target_image, return_tensors='pt')['pixel_values'].to(device)  # shape: [1, 3, 224, 224]

    # 加载并预处理一批图像
    images = [Image.open(p).convert("RGB") for p in image_paths]
    image_inputs = processor(images=images, return_tensors='pt')['pixel_values'].to(device)  # shape: [N, 3, 224, 224]

    with torch.no_grad():
        image_features = model.get_image_features(pixel_values=image_inputs)
        target_features = model.get_image_features(pixel_values=target_input)

    # 归一化
    image_features = image_features / image_features.norm(dim=1, keepdim=True)
    target_features = target_features / target_features.norm(dim=1, keepdim=True)

    # 批量计算余弦相似度
    similarities = torch.matmul(image_features, target_features.T).squeeze(1)  # shape: [N]
    return similarities.tolist()  # 返回列表

# ====== 修改这里的路径 ======
image_folder = '/media/dell/DATA/RK/InstantStyle-main/定量实验/output/暂存/our_sdxl'  # 替换成你的图像路径
target_folder = '/media/dell/DATA/RK/InstantStyle-main/定量实验/风格参考图像'  # 替换成你的目标图像路径
clip_model_path = '/media/dell/DATA/RK/pretrained_model/clip-vit-base-patch32'

# ====== 加载模型（只执行一次） ======
model, processor, device = load_clip_model(clip_model_path)

# 分组图像
group_images = defaultdict(list)

for filename in os.listdir(image_folder):
    if filename.endswith('.jpg') or filename.endswith('.png'):
        name = filename.split('.')[0]
        if '_' in name:
            group_id, idx = name.split('_')
            if group_id.isdigit() and idx.isdigit():
                group_images[int(group_id)].append((int(idx), os.path.join(image_folder, filename)))

# 排序图像
for group_id in group_images:
    group_images[group_id] = sorted(group_images[group_id], key=lambda x: x[0])

# 计算每组的相似度均值
all_group_scores = []

for group_id in sorted(group_images.keys()):
    image_paths = [path for _, path in group_images[group_id]]

    # 目标图像路径：如 1.jpg, 2.jpg ...
    target_path = os.path.join(target_folder, f"{group_id}.jpg")
    if not os.path.exists(target_path):
        print(f"❌ 缺少目标图像: {target_path}，跳过该组")
        continue

    try:
        scores = compute_batch_clip_similarities(
            image_paths=image_paths,
            target_image_path=target_path,
            model=model,
            processor=processor,
            device=device
        )
        avg_score = sum(scores) / len(scores)
        all_group_scores.append(avg_score)
        print(f"✅ Group {group_id:02d} - Avg CLIP Similarity: {avg_score:.4f}")
    except Exception as e:
        print(f"❌ 处理组 {group_id} 出错: {e}")

# 总体平均
overall_avg = sum(all_group_scores) / len(all_group_scores)
print(f"\n�� Total Average CLIP Similarity Across All Groups: {overall_avg:.4f}")

3.2CLIP-T

# clip_utils.py

from transformers import AutoModel, AutoProcessor, AutoTokenizer
from PIL import Image
import torch

def load_clip_text_model(clip_model_path):
    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
    model = AutoModel.from_pretrained(clip_model_path).to(device)
    processor = AutoProcessor.from_pretrained(clip_model_path)
    tokenizer = AutoTokenizer.from_pretrained(clip_model_path)
    return model, processor, tokenizer, device

def compute_clip_score_with_model(image_path, text, model, processor, tokenizer, device):
    # 图像预处理
    image = Image.open(image_path).convert('RGB')
    image_inputs = processor(images=image, return_tensors='pt').to(device)

    # 文本预处理
    text_inputs = tokenizer(text, return_tensors='pt', padding=True).to(device)

    # 特征提取
    with torch.no_grad():
        image_features = model.get_image_features(**image_inputs)
        text_features = model.get_text_features(**text_inputs)

    # 归一化
    image_features = image_features / image_features.norm(dim=1, keepdim=True)
    text_features = text_features / text_features.norm(dim=1, keepdim=True)

    # 相似度
    score = (image_features @ text_features.T).item()
    return score
import os
from collections import defaultdict

# 设置路径
image_folder = '/media/dell/DATA/RK/InstantStyle-main/定量实验/output/暂存/our_sdxl'
clip_model_path = '/media/dell/DATA/RK/pretrained_model/clip-vit-base-patch32'

# 文本描述列表
text_list = [
    "A ballerina spinning under a spotlight on stage.",
    "A man riding a unicycle through a marketplace.",
    "A woman practicing tai chi at sunrise.",
    "A child leaping into a puddle on a rainy day.",
    "A traveler photographing wildlife in the savannah.",
    "A person climbing a steep mountain trail with a backpack.",
    "A bustling street market filled with colorful stalls.",
    "A nighttime carnival with glowing rides and food trucks.",
    "A quiet family dinner under warm yellow lights.",
    "A futuristic control room with glowing interfaces.",
    "A stormy beach with waves crashing against rocks.",
    "A flamingo standing in shallow water at sunset.",
    "A polar bear walking across icy terrain.",
    "A monkey swinging from tree to tree in a jungle.",
    "A chameleon blending into colorful leaves.",
    "A school of fish swimming through a coral reef.",
    "A steaming cup of coffee on a windowsill.",
    "A pair of worn-out sneakers beside a doormat.",
    "An open book with handwritten notes in the margins.",
    "A hanging lightbulb glowing in a dark attic.",
    "An ancient stone bridge over a misty river.",
    "A modern glass skyscraper reflecting clouds.",
    "A wooden cabin in a snow-covered forest.",
    "A spiral staircase in an old library.",
    "A desert city with domed buildings and narrow alleys.",
    "A floating island with waterfalls cascading into the void.",
    "A glowing portal in the middle of a forest.",
    "A city built entirely from musical instruments.",
    "A dreamscape with floating clocks and melting trees.",
    "A mechanical bird flying through a sky of gears."
]

# 加载模型（一次）
model, processor, tokenizer, device = load_clip_text_model(clip_model_path)

# 组织图像：group_id => [(index, path)]
group_images = defaultdict(list)
for filename in os.listdir(image_folder):
    if filename.endswith('.jpg') or filename.endswith('.png'):
        group_id, index = filename.split('.')[0].split('_')
        group_images[int(group_id)].append((int(index), os.path.join(image_folder, filename)))

# 排序
for group_id in group_images:
    group_images[group_id] = sorted(group_images[group_id], key=lambda x: x[0])

# 主循环
all_group_scores = []

for group_id in sorted(group_images.keys()):
    total_score = 0.0
    count = 0
    for idx, img_path in group_images[group_id]:
        if idx > len(text_list):
            continue

        try:
            score = compute_clip_score_with_model(
                image_path=img_path,
                text=text_list[idx - 1],
                model=model,
                processor=processor,
                tokenizer=tokenizer,
                device=device
            )
            total_score += score
            count += 1
        except Exception as e:
            print(f"⚠️ 图像 {img_path} 处理失败: {e}")

    avg_score = total_score / count if count > 0 else 0
    all_group_scores.append(avg_score)
    print(f"✅ Group {group_id} - Avg CLIP Score: {avg_score:.4f}")

# 总体平均
overall_avg = sum(all_group_scores) / len(all_group_scores)
print(f"\n�� Total Average CLIP Score (All Groups): {overall_avg:.4f}")

3.3DONO

import os
from PIL import Image
from transformers import AutoImageProcessor, AutoModel
import torch
import torch.nn.functional as F
from collections import defaultdict
from tqdm import tqdm

# ==== 参数设置 ====
generated_image_folder = '/media/dell/DATA/RK/InstantStyle-main/定量实验/output/暂存/our_sdxl'
real_image_folder = '/media/dell/DATA/RK/InstantStyle-main/定量实验/风格参考图像'
model_path = '/media/dell/DATA/RK/pretrained_model/dinov2-base'
batch_size = 16

# ==== 设备 ====
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

# ==== 加载模型 ====
processor = AutoImageProcessor.from_pretrained(model_path)
model = AutoModel.from_pretrained(model_path).to(device)
model.eval()

# ==== 按组组织图像 ====
group_images = defaultdict(list)

for filename in os.listdir(generated_image_folder):
    if filename.endswith('.jpg') or filename.endswith('.png'):
        name = os.path.splitext(filename)[0]
        if '_' in name:
            group_id, idx = name.split('_')
            group_images[int(group_id)].append(os.path.join(generated_image_folder, filename))

# ==== 按照 group_id 排序 ====
for gid in group_images:
    group_images[gid].sort()

# ==== 计算每组的平均 DINO 相似度 ====
group_scores = []

for group_id in sorted(group_images.keys()):
    gen_image_paths = group_images[group_id]
    real_image_path = os.path.join(real_image_folder, f'{group_id}.jpg')

    # 加载并处理目标图像（仅一次）
    with torch.no_grad():
        real_image = Image.open(real_image_path).convert("RGB")
        real_inputs = processor(images=real_image, return_tensors="pt").to(device)
        real_features = model(**real_inputs).last_hidden_state.mean(dim=1)  # (1, D)
        real_features = F.normalize(real_features, dim=1)

    # 批量处理生成图像
    similarities = []
    for i in range(0, len(gen_image_paths), batch_size):
        batch_paths = gen_image_paths[i:i + batch_size]
        batch_images = [Image.open(p).convert("RGB") for p in batch_paths]

        with torch.no_grad():
            batch_inputs = processor(images=batch_images, return_tensors="pt").to(device)
            batch_features = model(**batch_inputs).last_hidden_state.mean(dim=1)
            batch_features = F.normalize(batch_features, dim=1)

        # Cosine similarity: (B, D) @ (D, 1) => (B, 1)
        batch_sim = (batch_features @ real_features.T).squeeze(1)
        batch_sim = (batch_sim + 1) / 2  # 归一化到 [0, 1]
        similarities.extend(batch_sim.tolist())

    avg_sim = sum(similarities) / len(similarities)
    group_scores.append(avg_sim)
    print(f"✅ Group {group_id} - Avg DINO Score: {avg_sim:.4f}")

# ==== 总平均 ====
overall_avg = sum(group_scores) / len(group_scores)
print(f"\n�� Total Average DINO Score (All Groups): {overall_avg:.4f}")