Text-to-Image Research

作者: 晓智AI | 来源:发表于2018-09-08 23:14 被阅读0次

研究背景

图像合成方向研究

学习参考

AttnGAN论文
 AttnGAN算法代码

CVPR2016
CVPR2017
论文翻译学习
 SSD中英对照
 Yolo9000中英对照

pytorch手册参考 github

Overleaf v2
Overleaf cn

Image Generation from Scene Graphs论文
 Image Generation from Scene Graphs代码

环境配置

Ubuntu 16.04 LTS

conda install pytorch

aria2c https://conda.anaconda.org/pytorch/linux-64/pytorch-0.4.1-py36_cuda9.0.176_cudnn7.1.2_1.tar.bz2

conda install cuda

(ssdpytorch) ouc@ouc:~/data1/liuhongzhi$ conda install cudatoolkit==8.0

conda install cudnn

(ssdpytorch) ouc@ouc:~/data1/liuhongzhi$ conda install cudnn

install lua package

git clone https://github.com/qassemoquab/stnbhwd.git
cd stnbhwd
luarocks install *.rockspec
......
stnbhwd scm-1 is now built and installed in /home/ouc/data1/liuhongzhi/distro/install/ (license: MIT)

display scm-0 is now built and installed in /home/ouc/data1/liuhongzhi/distro/install/ (license: MIT)

使用luarocks list查看是否安装成功

(gawwn) ouc@ouc:~/data1/liuhongzhi/GAWWN$ luarocks list
Warning: Failed loading manifest for /home/ouc/.luarocks/lib/luarocks/rocks: /home/ouc/.luarocks/lib/luarocks/rocks/manifest: No such file or directory

Installed rocks:
----------------

argcheck
   scm-1 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

cudnn
   scm-1 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

cunn
   scm-1 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

cutorch
   scm-1 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

cwrap
   scm-1 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

dok
   scm-1 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

env
   scm-1 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

fn
   0-0 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

gnuplot
   scm-1 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

graph
   scm-1 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

hdf5
   20-0 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

image
   1.1.alpha-0 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

logroll
   0-0 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

lua-cjson
   2.1devel-1 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

luaffi
   scm-1 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

luafilesystem
   1.6.3-1 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

matio
   scm-1 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

moses
   1.6.1-1 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

nn
   scm-1 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

nngraph
   scm-1 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

nnx
   0.1-1 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

optim
   1.0.5-0 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

paths
   scm-1 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

penlight
   scm-1 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

pprint
   0-0 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

qtlua
   scm-1 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

qttorch
   scm-1 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

stnbhwd
   scm-1 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

sundown
   scm-1 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

sys
   1.1-0 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

threads
   scm-1 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

torch
   scm-1 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

totem
   0-0 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

trepl
   scm-1 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

util
   0-0 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

xlua
   1.0-0 (installed) - /home/ouc/data1/liuhongzhi/distro/install/lib/luarocks/rocks

常用代码

1、生成AttnGAN需要的*.pickle文件

import pickle
#fi =open('huizong10000list.txt','w')
content = []
with open('vallist.txt', 'r') as f:
   for line in f.readlines():
       content.append(line.strip('\n'))
print(content)
#print(content,file=fi)
#fi.close()
output = open('vallist5000.pickle', 'wb')
pickle.dump(content, output, protocol=2)
output.close()

2、其他代码待整理

#导入模块
import pickle
fi =open("out.txt", "w")
#打开二进制文件
f = open('./coco/captions.pickle','rb')
n = pickle.load(f)
print("%3d %0.2f" % (year, principal), file = fi)            

for i in range(n):      #文件中有多少个对象就循环多个少次
    x = pickle.load(f)   #依次读取文件中对象
    print(x)
    #str = "a string to print to file"
    #file =open('out.txt','w')
    #print >>f,str
    #file.close()
f.close()


import pickle
#fi =open('out.txt','w')
picklefile=open('./train/filenames.pickle','rb')
data=pickle.load(picklefile)
#print (data,file=fi)
print (data)


import pickle



#导入模块
import pickle

with open('huizong.txt','r',encoding='utf-8') as src,open('huizong.pickle','wb') as dest:
    lines = src.readlines()  #把源文件中的所有内容都读取到 lines 列表中
    pickle.dump(len(lines),dest)  #先写入对象个数
    for line in lines:
        pickle.dump(line,dest)

#上面就是已经把数据写入到二进制文件中了，下面从二进制文件中反序列化对象
with open('huizong.pickle','rb') as f:
    n = pickle.load(f)
    for i in range(n):
        print(pickle.load(f))



import pickle
#fi =open('out.txt','w')
picklefile=open('huizong.pickle','rb')
data=pickle.load(picklefile)
#print (data,file=fi)
print (data)


import pickle
with open('huizong.txt', 'r') as fr:
    data = fr.read()
    print(data)
with open('huizong.pickle', 'wb') as fw:
    pickle.dump(data, fw, pickle.HIGHEST_PROTOCOL)
with open('huizong.pickle', 'rb') as fr:
    data = pickle.load(fr)
    print(data)



file = open('huizong2.txt', 'w')
for i in range (len('huizong.txt')):
  file.write(content[i]+'\n')
file.close()

f = open('huizong.pickle', 'wb')
test_content = text_read('huizong.txt')
def text_read(filename):
    # Try to read a txt file and return a list.Return [] if there was a mistake.
    try:
        file = open(filename,'r')
    except IOError:
        error = []
        return error
    content = file.readlines()

    for i in range(len(content)):
        content[i] = content[i][:len(content[i])-1]

    file.close()
    return content
print(test_content,file=f)



import pickle
with open('huizong2.txt', 'r') as fr:
    data = fr.read()
    print(data)
with open('huizong.pickle', 'wb') as fw:
    pickle.dump(data, fw, protocol=2)
with open('huizong.pickle', 'rb') as fr:
    data = pickle.load(fr)
    print(data)

import pickle
with open('huizong.pickle', 'rb') as fr:
    data = pickle.load(fr)
    print(data)

import pickle
with open('./train/filenames.pickle', 'rb') as fr:
    data = pickle.load(fr)
    print(data)

fi =open("out.txt", "w")
import pickle
with open('./train/filenames.pickle', 'rb') as fr:
    data = pickle.load(fr)
    print(data,file=fi)




import pickle
with open('huizong2.txt', 'r') as fr:
    data = fr.read()
    print(data)
with open('huizong.pickle', 'wb') as fw:
    pickle.dump(data, fw, protocol=2)
with open('huizong.pickle', 'rb') as fr:
    data = pickle.load(fr)
    print(data)

import pickle
selfref_list = ['COCO_train2014_000000487025', 'COCO_train2014_000000526896']
selfref_list.append(selfref_list)
output = open('list2.pickle', 'wb')
pickle.dump(selfref_list, output, protocol=2)
output.close()


import pickle
selfref_list = ['COCO_train2014_000000000025','COCO_train2014_000000526896']
output = open('list2.pickle', 'wb')
pickle.dump(selfref_list, output, protocol=2)
output.close()


import sys
result=[]
with open('huizong10000.txt','r') as f:
    for line in f:
        result.append(list(line.strip('\n').split(',')))
print(result)

import sys
result1=[]
with open('huizong10000.txt','r') as f:
    for line in f:
        result1.append(line.strip('\n').split(','))
print(result1)

#coding=utf-8
 
content = []

with open('huizong10000.txt', 'r') as f:
    for line in f.readlines():
        content.append([line.strip('\n')])
         
print(content)


import pickle
fi =open('huizong10000list.txt','w')
content = []  
with open('huizong10000.txt', 'r') as f:
   for line in f.readlines():
       content.append(line.strip('\n'))         
print(content,file=fi)
fi.close()
output = open('list10000.pickle', 'wb')
pickle.dump(content, output, protocol=2)
output.close()

henry@henry-System-Product-Name:~/Files/AttnGAN/data/coco/newpickle$ ipython
Python 3.7.0 (default, Jun 28 2018, 13:15:42) 
Type 'copyright', 'credits' or 'license' for more information
IPython 6.5.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import pickle
   ...: fi =open('huizong10000list.txt','w')
   ...: content = []  
   ...: with open('huizong10000.txt', 'r') as f:
   ...:    for line in f.readlines():
   ...:        content.append(line.strip('\n'))         
   ...: print(content,file=fi)
   ...: fi.close()
   ...: output = open('list10000.pickle', 'wb')
   ...: pickle.dump(content, output, protocol=2)
   ...: output.close()



import pickle
selfref_list = ['COCO_train2014_000000151351', 'COCO_train2014_000000151352', 'COCO_train2014_000000151353', 'COCO_train2014_000000151364', 'COCO_train2014_000000151371', 'COCO_train2014_000000151375', 'COCO_train2014_000000151382', 'COCO_train2014_000000151405', 'COCO_train2014_000000151406', 'COCO_train2014_000000151408', 'COCO_train2014_000000151414', 'COCO_train2014_000000151455', 'COCO_train2014_000000151466', 'COCO_train2014_000000151483', 'COCO_train2014_000000151486', 'COCO_train2014_000000151497', 'COCO_train2014_000000151499', 'COCO_train2014_000000151523', 'COCO_train2014_000000151564', 'COCO_train2014_000000151566', 'COCO_train2014_000000151567', 'COCO_train2014_000000151573', 'COCO_train2014_000000151577', 'COCO_train2014_000000151587', 'COCO_train2014_000000151594', 'COCO_train2014_000000151599', 'COCO_train2014_000000151609', 'COCO_train2014_000000151611', 'COCO_train2014_000000151619', 'COCO_train2014_000000151637', 'COCO_train2014_000000151646', 'COCO_train2014_000000151655', 'COCO_train2014_000000151658', 'COCO_train2014_000000151678', 'COCO_train2014_000000151694', 'COCO_train2014_000000151702', 'COCO_train2014_000000151731', 'COCO_train2014_000000151756', 'COCO_train2014_000000151757', 'COCO_train2014_000000151761', 'COCO_train2014_000000151764', 'COCO_train2014_000000151787', 'COCO_train2014_000000151840', 'COCO_train2014_000000151848', 'COCO_train2014_000000151862', 'COCO_train2014_000000151869', 'COCO_train2014_000000151893', 'COCO_train2014_000000151900', 'COCO_train2014_000000151934', 'COCO_train2014_000000151959', 'COCO_train2014_000000151979', 'COCO_train2014_000000152003', 'COCO_train2014_000000152036', 'COCO_train2014_000000152038', 'COCO_train2014_000000152056', 'COCO_train2014_000000152060', 'COCO_train2014_000000152084', 'COCO_train2014_000000152099', 'COCO_train2014_000000152108', 'COCO_train2014_000000152162', 'COCO_train2014_000000152197', 'COCO_train2014_000000152209', 'COCO_train2014_000000152261', 'COCO_train2014_000000152265', 'COCO_train2014_000000152269', 'COCO_train2014_000000152273', 'COCO_train2014_000000152275', 'COCO_train2014_000000152294', 'COCO_train2014_000000152309', 'COCO_train2014_000000152354', 'COCO_train2014_000000152389', 'COCO_train2014_000000152397', 'COCO_train2014_000000152403', 'COCO_train2014_000000152405', 'COCO_train2014_000000152429', 'COCO_train2014_000000152431', 'COCO_train2014_000000152434', 'COCO_train2014_000000152461', 'COCO_train2014_000000152483', 'COCO_train2014_000000152488']
output = open('list10000.pickle', 'wb')
pickle.dump(selfref_list, output, protocol=2)
output.close()

3、Python 批量改变图像大小size代码

import os
from PIL import Image
import sys


#获取path目录下的所有文件
def get_imlist(path):
    return[os.path.join(path,f) for f in os.listdir(path)]

def change_size(path):
    directorys=get_imlist(path)
    print("start directorys")
    count = 0
    for directory in directorys:
                #不是图片文件就跳过
        #if not(directory.endswith('j.jpg') or directory.endswith('b.bmp')):
        #   pass
        #else:

            count = count +1
            print(count)
            print("start!")
            img=Image.open(directory)
            s="/"
                    #获取文件名（含后缀）
            oimage_name=directory[directory.rfind(s)+1:]
             #获取原图的宽度和高度
            (oimage_width,oimage_height)=img.size

            if oimage_width==256 and oimage_height==256:
                to_save=path+'/256_256/'+oimage_name
                img.save(to_save)
               #移除原图
                os.remove(directory)
            else:
                to_save='/home/henry/Files/ICCV2019/cocostuffapi/PythonAPI/text2image/pix2pix/'+'/256/'+oimage_name  # 保存路径
                new_width=256
                new_height=256
                out=img.resize((new_width,new_height),Image.ANTIALIAS)
                out.save(to_save)
                os.remove(directory)

change_size("/home/henry/Files/ICCV2019/cocostuffapi/PythonAPI/text2image/pix2pix/trainresize")   # 原始图像路径

4、根据txt中文件名，将指定文件复制到指定路径代码

import time     
import os  
import shutil
 
def re_mycopyfile(srcfile,dstfile,num):
    #name_long=16
    l=len(str(num))
    zero='00000000'
    newname = srcfile[-16:-4]
    if not os.path.isfile(srcfile):
        print "%s not exist!"%(srcfile)
    else:
        #fpath,fname=os.path.split(dstfile)    #分离文件名和路径
        if not os.path.exists(dstfile):
            os.makedirs(dstfile) #创建路径
        #dstfile=dstfile+zero[:name_long-l-1]+str(num)+'.txt'
        dstfile = dstfile+str(newname)+'.txt'
        print dstfile             
        shutil.copyfile(srcfile,dstfile)      #复制文件
        print "copy %s -> %s"%(srcfile,dstfile)
 
 
 
if __name__ == '__main__':
    path1="/home/henry/Files/ICCV2019/cocostuffapi/PythonAPI/pic256.txt"
    path2="/home/henry/Files/ICCV2019/cocostuffapi/PythonAPI/train2017all/"
    path3="/home/henry/Files/ICCV2019/cocostuffapi/PythonAPI/train2017/"
    path4="/home/henry/Files/ICCV2019/cocostuffapi/PythonAPI/trainnew.txt"
 
    begin=0
    count=begin
    with open(path1,'r')as f:
        for line in f:
            line=line.split('\n')
            print line[0]
            srcfile = path2+str(line[0])
            print srcfile
            count=count+1
            print count
            dstfile=path3
            re_mycopyfile(srcfile,dstfile,count)
 
    count=begin
    name_long=6
    l=len(str(count+1))
    zero='00000000'

    with open(path1,'r')as f:
        for line in f:
            count=count+1
            out_words=line.split('/')
            #out_words[-1]=zero[:name_long-l-1]+str(count)+'.txt'
            out_words[-1] = zero[:name_long - l - 1] + str(count) + '.txt'
            with open(path4,'a+') as fp:
                fp.write("/".join(out_words)+"\n")

5、ubuntu批量转换 png到jpg代码

----------- 从 PNG 转换到 JPG -----------
$ ls -1 *.png | xargs -n 1 bash -c 'convert "$0" "${0%.png}.jpg"'
----------- 从 JPG 转换到 PNG -----------
$ ls -1 *.jpg | xargs -n 1 bash -c 'convert "$0" "${0%.jpg}.png"'

-1 – 告诉 ls 每行列出一个图像名称的选项标识
-n – 指定最多参数个数，例子中为 1
-c – 指示 bash 运行给定的命令
${0%.png}.jpg – 设置新转换的图像文件的名字，% 符号用来删除源文件的扩展名

问题总结

1、ImportError: No module named torchvision
ImportError: No module named torchvision

2、ImportError: No module named yaml
解决办法

pip install pyyaml

3、Conda常用命令整理
Conda常用命令整理
4 conda
ubuntu利用conda创建虚拟环境并安装cuda cudnn
5 find 查找文件并复制到指定路径
src_dir 源目录
dst_dir 目标目录
access.log.2011102[2-6]* 文件名的正则表达式，获取文件的条件

方法1：

find src_dir -name "access.log.2011102[2-6]*" -exec cp {} dst_dir /

find src_dir -name "access.log.2011102[2-6]*" -exec scp {} 用户名@主机ip:dst_dir /

方法2：

find src_dir -name "access.log.2011102[2-6]*" |xargs -i cp {} dst_dir

find src_dir -name "access.log.2011102[2-6]*" |xargs -I {} cp {} dst_dir

find src_dir -name "access.log.2011102[2-6]*" |xargs -I {} scp {} 用户名@主机ip:dst_dir

方法3：

find ./ -path '/tmp/mnt/disk1/ignore' -prune -o /( -name '*' ! -name "*.tmp" /) | xargs cp "目的目录" "{}" /

6、OpenCV版本报错

ImportError: No module named 'cv2'

版本错误
AttributeError: 'module' object has no attribute 'CV_LOAD_IMAGE_COLOR'\

原因版本问题，opencv3使用
IMREAD_COLOR 代替'CV_LOAD_IMAGE_COLOR'

安装OpenCV

Run the following command:

conda install -c https://conda.binstar.org/menpo opencv

conda install -c https://conda.binstar.org/menpo opencv3

conda config --add channels menpo

conda install opencv (or opencv3)

conda install -c clinicalgraphics vtk

CycleGAN

visdom

启动

python3 -m visdom.server

初始化

from visdom import Visdom
import numpy as np
viz = Visdom(env='loss')
x,y=0,0.2
win = viz.line( X=np.array([x]), Y=np.array([y]), opts=dict(title='loss'))

Visdom(server='http://localhost', endpoint='events', port=8097, ipv6=True, http_proxy_host=None, http_proxy_port=None, env='main', send=True, raise_exceptions=None, use_incoming_socket=True)

3.更新参数

viz.line(X=np.array([x_new]),Y=np.array([y_new]), win=win,update='append')

必备深度学习知识

给定一副测试图片输入(N,C,H,W)
参数 W 是图像的宽度，H 是高度，C 是通道的个数；彩色图像中 C = 3，灰度图像中C = 1
data_format 设置为[ N C H W ]时，排列顺序为 [batch, channels, height, width]，[批，通道数，高度，宽度]

Text-to-Image Research

研究背景

学习参考

环境配置

常用代码

1、生成AttnGAN需要的*.pickle文件

2、其他代码待整理

3、Python 批量改变图像大小size代码

4、根据txt中文件名，将指定文件复制到指定路径代码

5、ubuntu批量转换 png到jpg代码

问题总结

必备深度学习知识

相关文章

网友评论

延伸阅读

深度阅读

栏目导航

热点阅读