Create Anaconda virtual environment with python 3.5

conda create --name opencv3 python=3.5 anaconda

Install OpenCV3 by homebrew

brew install opencv3 --with-python3

Link OpenCV3

Link site-package from brew to conda
For example:

echo /usr/local/opt/opencv3/lib/python3.6/site-packages >> /Users/rabbitlee/anaconda/envs/opencv3/lib/python3.5/site-packages/opencv3.pth

And one more thing
Change the so file name
For example:

cd /usr/local/opt/opencv3/lib/python3.5/site-packages
mv cv2.cpython-35m-darwin.so ./cv2.so

Testing

Try to use cv2 package:

source activate opencv3
python -c "import cv2"

If there is no error, my congratulations — we did it!

Reference

[Step 5 is important](https://medium.com/@satchitananda/setting-up-opencv-for-python-3-on-macos-sierra-with-5-easy-steps-647b64c5e0c9