Tutorial

Let us start an interactive session (e.g. with python or ipython) and import Theano.

>>> from theano import *

Several of the symbols you will need to use are in the tensor subpackage of Theano. Let us import that subpackage under a handy name like T (the tutorials will frequently use this convention).

>>> import theano.tensor as T

If that succeeded you are ready for the tutorial, otherwise check your installation (see Installing Theano).

Throughout the tutorial, bear in mind that there is a Glossary as well as index and modules links in the upper-right corner of each page to help you out.

• Python tutorial
• NumPy refresher
  • Matrix conventions for machine learning
  • Broadcasting
• Baby Steps - Algebra
  • Adding two Scalars
  • Adding two Matrices
  • Exercise
• More Examples
  • Logistic Function
  • Computing More than one Thing at the Same Time
  • Setting a Default Value for an Argument
  • Using Shared Variables
  • Using Random Numbers
    • Brief Example
    • Seeding Streams
    • Sharing Streams Between Functions
    • Copying Random State Between Theano Graphs
    • Other Random Distributions
    • Other Implementations
  • A Real Example: Logistic Regression
• Graph Structures
  • Theano Graphs
  • Automatic Differentiation
  • Optimizations
• Printing/Drawing Theano graphs
  • Pretty Printing
  • Debug Print
  • Picture Printing of Graphs
  • Interactive Graph Visualization
• Derivatives in Theano
  • Computing Gradients
  • Computing the Jacobian
  • Computing the Hessian
  • Jacobian times a Vector
    • R-operator
    • L-operator
  • Hessian times a Vector
  • Final Pointers
• Configuration Settings and Compiling Modes
  • Configuration
  • Exercise
  • Mode
  • Linkers
  • Using DebugMode
  • ProfileMode
    • Creating a ProfileMode Instance
    • Compiling your Graph with ProfileMode
    • Retrieving Timing Information
• Loading and Saving
  • The Basics of Pickling
  • Short-Term Serialization
  • Robust Serialization
  • Long-Term Serialization
• Conditions
  • IfElse vs Switch
• Loop
  • Scan
  • Exercise
• Sparse
  • Compressed Sparse Format
    • Which format should I use?
  • Handling Sparse in Theano
    • To and Fro
    • Properties and Construction
    • Structured Operation
    • Gradient
• Using the GPU
  • CUDA backend
    • Testing Theano with GPU
    • Returning a Handle to Device-Allocated Data
    • What Can Be Accelerated on the GPU
    • Tips for Improving Performance on GPU
    • GPU Async capabilities
    • Changing the Value of Shared Variables
      • Exercise
  • GpuArray Backend
    • Testing Theano with GPU
    • Returning a Handle to Device-Allocated Data
    • What Can be Accelerated on the GPU
    • GPU Async Capabilities
  • Software for Directly Programming a GPU
  • Learning to Program with PyCUDA
    • Exercise
    • Exercise
  • Note
• Using multiple GPUs
  • Defining the context map
  • A simple graph on two GPUs
  • Explicit transfers of data
• PyCUDA/CUDAMat/Gnumpy compatibility
  • PyCUDA
    • Transfer
    • Compiling with PyCUDA
    • Theano Op using a PyCUDA function
  • CUDAMat
  • Gnumpy
• Understanding Memory Aliasing for Speed and Correctness
  • The Memory Model: Two Spaces
  • Borrowing when Creating Shared Variables
  • Borrowing when Accessing Value of Shared Variables
    • Retrieving
    • Assigning
  • Borrowing when Constructing Function Objects
• How Shape Information is Handled by Theano
  • Shape Inference Problem
  • Specifing Exact Shape
  • Future Plans
• Debugging Theano: FAQ and Troubleshooting
  • Isolating the Problem/Testing Theano Compiler
  • Interpreting Error Messages
  • Using Test Values
  • “How do I Print an Intermediate Value in a Function?”
  • “How do I Print a Graph?” (before or after compilation)
  • “The Function I Compiled is Too Slow, what’s up?”
  • “Why does my GPU function seem to be slow?”
  • “How do I Step through a Compiled Function?”
  • How to Use pdb
  • Dumping a Function to help debug
• Dealing with NaNs
  • Check Superparameters and Weight Initialization
  • Run in NanGuardMode, DebugMode, or MonitorMode
  • Numerical Stability
  • Algorithm Related
• Profiling Theano function
• Extending Theano
  • Theano Graphs
  • Op Structure
  • Op Example
  • Example for properties of a Op
  • How To Test it
    • Basic Tests
    • Testing the infer_shape
    • Testing the gradient
    • Testing the Rop
    • Testing GPU Ops
  • Running Your Tests
    • theano-nose
    • nosetests
    • In-file
  • Exercise
  • as_op
    • as_op Example
    • Exercise
  • Random numbers in tests
  • Documentation
  • Final Note
• Extending Theano with a C Op
  • Python C-API
    • Reference counting
  • NumPy C-API
    • NumPy data types
    • NumPy ndarrays
    • Accessing NumPy ndarrays’ data and properties
    • Creating NumPy ndarrays
  • Methods the C Op needs to define
  • Simple C Op example
  • More complex C Op example
  • Alternate way of defining C Ops
    • Main function
    • Macros
    • Support code
  • Final Note
• Python Memory Management
  • Basic Objects
  • Internal Memory Management
  • Pickle
• Multi cores support in Theano
  • BLAS operation
  • Parallel element wise ops with OpenMP
• Frequently Asked Questions
  • How to update a subset of weights?