|
| 1 | +# Pretrained BigBiGAN models |
| 2 | + |
| 3 | +We have released pretrained BigBiGAN models used for unsupervised image |
| 4 | +generation and representation learning, as described in our July 2019 |
| 5 | +tech report, |
| 6 | +["Large Scale Adversarial Representation Learning"](https://arxiv.org/abs/1907.02544) [1]. |
| 7 | + |
| 8 | +The pretrained models are available for use via [TF Hub](https://tfhub.dev/s?publisher=deepmind&q=bigbigan). |
| 9 | +The release includes two BigBiGAN models with different encoder architectures: |
| 10 | + |
| 11 | +* Small encoder (ResNet-50): [https://tfhub.dev/deepmind/bigbigan-resnet50/1](https://tfhub.dev/deepmind/bigbigan-resnet50/1) |
| 12 | +* Large encoder (RevNet-50 x4): [https://tfhub.dev/deepmind/bigbigan-revnet50x4/1](https://tfhub.dev/deepmind/bigbigan-revnet50x4/1) |
| 13 | + |
| 14 | +See the TF Hub pages linked above for documentation and example usage of each module. |
| 15 | + |
| 16 | +## Demo (Colab) |
| 17 | + |
| 18 | +A Google Colab-based demo with example usage of the model functionality and sample visualization is available [here](//colab.research.google.com/github/tensorflow/hub/blob/master/examples/colab/bigbigan_with_tf_hub.ipynb). |
| 19 | + |
| 20 | +## Example use |
| 21 | +The snippet below demonstrates the use of the released TF Hub modules for |
| 22 | +image generation/reconstruction and encoder feature computation. |
| 23 | +(The [Colab demo](//colab.research.google.com/github/tensorflow/hub/blob/master/examples/colab/bigbigan_with_tf_hub.ipynb) |
| 24 | +includes more extensive documentation and visualizations.) |
| 25 | + |
| 26 | +```python |
| 27 | +# Load BigBiGAN module. |
| 28 | +module = hub.Module('https://tfhub.dev/deepmind/bigbigan-resnet50/1') # small encoder |
| 29 | +# module = hub.Module('https://tfhub.dev/deepmind/bigbigan-revnet50x4/1') # large encoder |
| 30 | + |
| 31 | +# Sample a batch of 8 random latent vectors (z) from the Gaussian prior. Then |
| 32 | +# call the generator on the latent samples to generate a batch of images with |
| 33 | +# shape [8, 128, 128, 3] and range [-1, 1]. |
| 34 | +z = tf.random.normal([8, 120]) # latent samples |
| 35 | +gen_samples = module(z, signature='generate') |
| 36 | + |
| 37 | +# Given a batch of 256x256 RGB images in range [-1, 1], call the encoder to |
| 38 | +# compute predicted latents z and other features (e.g. for use in downstream |
| 39 | +# recognition tasks). |
| 40 | +images = tf.placeholder(tf.float32, shape=[None, 256, 256, 3]) |
| 41 | +features = module(images, signature='encode', as_dict=True) |
| 42 | + |
| 43 | +# Get the predicted latent sample `z_sample` from the dict of features. |
| 44 | +# Other available features include `avepool_feat` and `bn_crelu_feat`, used in |
| 45 | +# the representation learning results. |
| 46 | +z_sample = features['z_sample'] # shape [?, 120] |
| 47 | + |
| 48 | +# Compute reconstructions of the input `images` by passing the encoder's output |
| 49 | +# `z_sample` back through the generator. Note that raw generator outputs are |
| 50 | +# half the resolution of encoder inputs (128x128). To get upsampled generator |
| 51 | +# outputs matching the encoder input resolution (256x256), instead use: |
| 52 | +# recons = module(z_sample, signature='generate', as_dict=True)['upsampled'] |
| 53 | +recons = module(z_sample, signature='generate') # shape [?, 128, 128, 3] |
| 54 | +``` |
| 55 | + |
| 56 | +## References |
| 57 | + |
| 58 | +[1] Jeff Donahue and Karen Simonyan. |
| 59 | +[Large Scale Adversarial Representation Learning](https://arxiv.org/abs/1907.02544). |
| 60 | +*arxiv:1907.02544*, 2019. |
0 commit comments