A new tree addon, part VI: continuing development

The latest version (0.2.14) includes a module written in C that implements some of the most time consuming functions. The result is a 20 - 30 % speedup for the generation of the branch skeleton. For now this is only available for 64 bit windows but on a different platform the addon will fall back on pure python implementations of these functions so cross platform portability is still guaranteed.

This is a quite noticable speedup: on my machine the generation of a tree consisting of 3000 branch segments went from 3.7s to 2.7s.

Implementation notes

The C extension module was compiled with Visual Studio 2013 and seems to work fine both with the official 2.70 distribution (compiled with MSVC 2008) and with the daily builds (compiled with MSVC 2013) but will only work with 64 versions of Blender. The Visual Studion solution is part of the spacetree repository so if people are interested in porting the module to other platforms they can have a go. Detailed notes on how to compile a native Python extension with Visual Studio 2013 were published in a separate article.

Undocumented stuff

This version of the addon also contains a first try at the implementation of branch shedding.

References

The space tree addon itself is introduced in a few articles on this blog and is available on GitHub. Relevant links are listed below:

Native Python extensions for Blender with Visual Studio 2013

Researching how to create a python module in C with Microsoft Visual Studio 2013 which could be used in Blender caused me quite a few headaches so in order to spare other people the same I wrote this article that illustrates the process in a step-by-step manner. The resulting solution is available on GitHub as well.

Prerequisites

When writing native extensions for Python it is said that it is vitally important to use exactly the same compiler that was used to compiler Blender itself (or rather its embedded Python). This is especially important on Windows where even different versions of Visual Studio may break compatibility. However, I have found that Visual Studio 2013 can be used to compile an extension that is usable both for Blender version compiled with msvc2008 and msvc2013.

Currently the daily builds for Blender are built with Visual Studio 2013. This can be verified by opening Blender and generating the system information from Help -> System Info. In the generated text file (available in the text editor) the line with the Python version should read something like: ...... 1800 64 bit .... The 1800 part is the interesting bit as it identifues the version of Visual Studio, where 1800 means MSVC 2013.

The official Blendet 2.70 release for windows is still built with MSVC 2008 (the version string in the system info will show 1500) but I have found no compatibility issues, but of course your mileage may vary. It is however extremely important to match the platform: a 64 bit extension will only work for 64 bit Blender!

Writing an extension module in C

Starting point when writing an extension is this page. It's a fairly complete but lengthy introduction and its content is platform independent. To actually compile C code into a dynamic library that can be used on Windows additional information is needed. Unfortunately some information on this page is wrong or incomplete (/export:initexample is not needed, the dll needs a .pyd extension) and as there are quite a few confusing steps it might be easier to have all the steps explicitely listed for our Blender scenario.

In the following walkthrough I'll assume that Blender is installed in C:\Blender and that we'll be using MVSC 2013 express edition for Desktop. I also assume you have checked out the Python library that Blender is built with, to C:\BlenderLibraries. We need the source for these libraries for the Python.h include files. (you can checkout these libraries with any subversion client from svn.blender.org, see here for instructions. Make sure you get the 64 bit versions if you are compiling for a 64 bit platform and notice that the folder is called win64_vc12 even though we'll be using msvc 2013)

Ten easy steps

The following steps are needed to create a simple extension. each step is explained in detail below.

  1. Create a new DLL project
  2. Add a new x64 platform if necessary
  3. Configure the extension to use
  4. Configure the include directory
  5. Configure the library directory
  6. Add the library dependencies
  7. Add your source code
  8. Build the project
  9. Copy the library to the Blender installation
  10. Test it from Blender

Create a new DLL project

  • file->new project
  • templates->Visual C++->Win32->Win32 Project
  • name: BlenderTest (can be placed anywhere, leave all default settings)
In the application wizard that opens select DLL and Empty project.

Add a new x64 platform if necessary

i=If you are working on/for a 64-bit platform you'll need to add a x64 platform to your solution.

  • click on the configuration manager [below the menu bar in the task bar]
  • add a new active platform [click on the part that probably displays Win32]
  • select x64 and leave all the default settings

Configure the extension to use

Although a native extension is just a DLL, Python wants it to have a .pyd extension:
In the properties of the BlenderTest project
for all configurations set
Configuration properties -> General the Target extension to .pyd

Configure the include directory

The compiler needs to be able to find the Python.h include file.
for all configurations
Configuration properties -> C/C++ -> General Add 'C:\BlenderLibraries\win64_vc12\python\include\python3.3' to Additional Include directories

Configure the library directory

The Python libraries need to found as well:
for all configurations
Configuration properties -> Linker -> General Add 'C:\BlenderLibraries\win64_vc12\python\lib' to Additional Library directories

Add the library dependencies

Each configuration is dependent on a different library:
for the Debug configuration
Linker -> Input Add 'python33_d.lib' to Additional Dependencies
for the Release configuration
Linker -> Input Add 'python33.lib' to Additional Dependencies

Add your source code

in the Solution explorer, right click the Source Files folder in the BlenderTest project and slect add -> new item. Choose C++ file and give it BlenderTest.cpp as its name (although you're free to choose anything you want)

Copy the example code (you might want to use this) in the editor

build the solution (a warning about an inconsistent dll linkage of round can be ignored)

Build the project

Select the release configuration and click Build -> Build Solution

The module will be in the subfolder BlenderTest\x64\Release and be called BlenderTest.pyd

Copy the library to the Blender installation

Copy BlenderTest.pyd to somewhere where Blender can find it, for example to the site-packages directory within Blenders installation directory (e.g. C:\Blender\2.70\python\lib\site-packages)

Test it from Blender

open the python console in Blender
import BlenderTest
BlenderTest.foo()

The System console (Window -> toggle system console) should show 'Hello World'

Book review: Blender Cycles Rendering and Lighting

I was asked by Packt Publishing to review Bernardo Iraci's book and I have to say I really enjoyed it.

Pros

  • cookbook style with many examples
  • thorough coverage of all things Cycles
  • gentle learning curve
  • some really great materials (especially carpaint and food)
  • addresses lighting as well
  • plenty of relevant external references

Cons

  • could do with some detail renders of the individual materials
  • doesn't cover volume shaders

The Cycles rendering engine has been available for some time now in Blender but it is still hot because it enables the artist to produce great results. However, its many options can overwhelm a person just starting out, so Bernardo's cookbook is a blessing both for beginners as well as for more accomplished artists.

The book is well written and covers almost everything from basic node setups to very complex ones, including subsurface scattering nodes and script nodes (It doesn't cover writing shaders in the Open Shading Language, for that you might consider my book Open Shading Language for Blender). The illustrations of the node setups are also very clear and although the book's style is a cookbook, the reasoning behind the creation of the materials is often explained as well which really helps to understand why the materials are set up the way the are.

I also like that the is a fair amount of information on lighting because now matter how good your materials are, without proper lighting your render won't shine. The book presents some useful lightning setups and covers both mesh lighting and HDRI lighting quite well.

The criticisms I have are minor indeed: the book doesn't cover the new volume shaders but this is of course not a fault of the author. Cycles evolves at a rapid pace and at the time of writing volume shaders were not yet available.

Also it would have been clearer of some of the many materials presented in this book would have closeup example renders; now there is just a larger overall render of a scene at the beginning of each chapter which doesn't show the smaller details very well. Of course the sample scenes are available to the reader so you can render them yourself but it would have made for an easier read.

All in all I really enjoyed reading this book and in my opinion it is one of the better ones on Blender published so far.

Conclusion

A well written book that delivers what it promises. Well worth its money.

A critical remark to all publishers

I read the pdf version of the book and potential buyers really should consider that too because the printed version is in black-and-white.

When will publishers realize that this kind of book should be printed in color or nit at all? Monochrome paper prints simply cannot convey the information presented in colorful illustrations very well and books on rendering are all about color!

So either invest in proper color plates as the print version already is quite expensive or just stick to e-book formats.

A new tree addon, part V: bug fixes and user contributions

The latest version (0.2.11) fixes several bugs and adds a a very nice collection of fruit objects to the bundled library.

Vitamins are good for you

BlenderArtists member Piet not only contributed a new bark material that adds the impression of snow lying on the branches but also provided a collection of several fruits: Apples, pears, plums, peaches, cherries as well as mistletoe and a birds nest. Thank you Piet! Now all we need is leaf materials to go with these fruits. I'll try to source some Creative Commons leaf textures but now that spring is getting underway (at least here in the Netherlands) I might create some of my own.

Bug fixes

This release also features some bug fixes: If you were using groups of objects to shape a crown and an object within this group wasn't a mesh object, the addon would throw an exception. Likewise, if you had an object selected before creating a tree, an exception was thrown if this object didn't have an edit mode (like the camera or an empty).

Observations


  • The bundled library is getting quite big, I am thinking about zipping it and uncompressing it on the fly.
  • The fruit objects (and the leaves as well) are not positioned perfectly (Most of the time they do not quite connect to a branch). I'll have to check if there origins are set correctly or if we need something else (fruit tree for example grow their fruits generally on small protrusion called spurs)
  • It might be sensible to add more than one object group from the library (for example acorns and mistletoe)

References

The space tree addon itself is introduced in a few articles on this blog and is available on GitHub. Relevant links are listed below:

A new tree addon, Part IV: volume rendering experiments with OSL

With the new volume shading options offered by Blenders OSL implementation it is tempting to try this on the trees generated by the space tree addon. Without adding any extra geometry or particles apart form a basic ahape a volume shader might give the impression of a much denser crown as shown in the image below:


This effect was achieved by inserting a scaled icosphere that covers about 90% of the crown interior (see below) and adding a volume shader to this icosphere that scatters and absorbs light in a non-uniform manner, i.e. the shader mimics a bunch of small scattered disks, which when seen from the distance add to the illusion of leaves. Note that we cannot do without all the leaves because volume scattering adds no specular reflections as rel leaves might do.


If you would like to know more about programming OSL you might be interested in my book "Open Shading Language for Blender". More on the availability of this book and a sample can be found on this page.

Shader code and example node setup

The code for this shader consists of the shader proper, which merely checks which randomly scattered point we are closest to and then calls the indisk function with the position of this closest point and a random direction. indisk checks whether we are inside a disk with its axis in some direction and returns 1 if this is indeed so. Note that the include at the start of the code refers to a file that is distriubted with Blender and contains a number of useful functions, including a number of Voronoi/Worley related ones.

#include "node_texture.h"

int indisk(
  point p,
  point c, float r, float h, vector d
){
  vector v = p - c;
  float a = dot(v,d);
  float lh = abs(length(a * d));
  if(lh > h){ return 0;}
  float lv = length(v);
  float lp = sqrt(lv*lv - lh*lh);
  if(lp > r){ return 0;}
  return 1;
}

vector randomdirection(point p){
  float t = M_2PI*noise("cell",p,1);  
  float u = 2*noise("cell",p,2)-1;  
  float s,c,a;  
  sincos(t,s,c);  
  a = sqrt(1-u*u);  
  float x = a*c;  
  float y = a*s;  
  float z = u;  
  return vector(x,y,z);
}

shader bits(
  point Pos = P,
  float Scale = 1,
  float Size = 1,
  float Height = 0.05,

  output float Fac = 0
){
  point p = Pos * Scale;
  point centers[4];
  float distances[4];
  voronoi(p, "Distance Squared", 0, distances, centers);
  if(indisk(p,centers[0],Size,Height,randomdirection(p))){
    Fac = 1;
  }
}
The node setup to use this shader as seen in the opening image of this article looks like this:

Discussion

Whether using a volume shader this way is really useful remains to be seen: rendering this way is still rather slow. Of course, getting a comparable dense crown with extra particles also slows down rendering: in my tests doubling the number of particles from 1000 to 2000 resulted in a render time that was actually slower than adding the volume shader. In other words, your mileage may vary but it might be worth to experiment.

References

The space tree addon itself is introduced in a few articles on this blog and is available on GitHub. Relevant links are listed below:

Book review: Blender compositing and post processing


Blender compositing and post processing
by Mythravarun Vepakomma.

pros

  • complete and concise reference of all compositor nodes
  • very clear illustrations explaining each node

cons

  • very few pages (about 100, not counting fringe content like index and front matter)
  • very few task oriented examples, just a short example of each node

I have mixed feelings about this book. Clearly the author knows his stuff and he sure knows how to make clear illustrations that explain each individual building block very well. However the book hardly offers any step by step walkthroughs for more complex, real world tasks. And solving complex tasks by stringing together node networks that each solve a particular task is what Blender's compositor excels at.

So the book is a fine reference but would have offered much more value for money if it had shown for example how to composite an actor in front of a green screen onto background footage from start to finish, including solutions for problems like uneven lighting of the green screen, masking hair, matching color, coping with reflected light, etc. and not just the simple demonstration of tbe alpha over node's options.

Conclusion: not a bad reference, produced quite well but I'm left with the feeling it could have been a lot more.

OSL Bevel Shader

In a BlenderArtists thread about ambient occlusion shaders the question was asked if OSL's ray tracing capabilities coudl be used to create a bevel shader, i.e. a s shader that mimics rounded edges on sharp meshes by manipulating the normal. I think the answer to this question is 'yes' and in this post I'' show a first implementation (Edit: the previous code did normalize the resulting normal, now it does. You can follow the discussion on this thread)


Sample scene, just a cube and an icosphere with sharp edges and some glossy material.

The effect of the bevel shader with Perturb = 0.1. Subtle but clearly noticable.

Setting Perturb = 1.0 is far less subtle but shows how much roundedness is possible. All images were produced with Divisions = 3, i.e. 27 sample rays (see below).

Code & theory

To simulate rounded edges choose a point a little below the surface and from this point we cast a set of rays that are all a little bit perturbed from the normal. The we determin the normal at these new intersections and average all normals found. This way when we are close to an edge the normals at the intersection with an adjacent face will be blended in. The illustration below shows what I mean:


At the shaded point we have a green surface normal. We descend below the surface along this normal (dashed red line) to the blue point. From this point we shoot rays (red) that are slightly perterbed from the normal. Some will hit the same face (and have the same normal as the shaded point (small blue arrows) some will hit the adjacent face with adifferent normal (small orange arraws). The calculated normal is the average of the normals at the hit points.

Note that this algorithm only works well for convex edges, because with concave edges random faces could be hit instead of the adjacent face if the angle between the faces is too sharp.

The code implementing this algorithm is pretty short:

#define BIG 1e6

shader edge(
  float Delta = 0.01,
  float Perturb = 0.001,
  int Divisions = 2,
  output float Fac = 0,
  output normal Normal = N
){
  vector origin = P-Delta*N;
  float count = 1;
  for(int x=0; x < Divisions; x++){
    for(int y=0; y < Divisions; y++){
      for(int z=0; z < Divisions; z++){
        vector d = noise("perlin",vector(x+0.1,y+0.1,z+0.1));
        vector np = N + Perturb * d;    
        if(trace(origin, np)){
            float hdb = 0;
            normal nb = 0;
            getmessage("trace","hitdist",hdb);
            // versions before 2.69.11 4-Mar-2014 crash on next statement
            getmessage("trace","N",nb);
            Normal -= nb;
            count += 1.0;
        }
      }
    }
  }
  Normal = normalize(Normal/count);
  Fac = count/(1+Divisions * Divisions * Divisions);
}

We choose an origin point a bit below the surface by descending along the normal [line 10], generate perturbed normals [line 15-16], trace these normals [line 16] and if we have a hit we retrieve the normal at the hit point [line 22]. Because we are below the surface/ inside the mesh this normal will point inward so we subtract it from our sum of normals (Normal). Finally we divide the sum of normals by the number of hits [line 30]. Note that we also get the hit distance [line 20] but don't use that value. We could check this againt some limit to reduce artifacts on concave meshes for instance.

The generation of perturbed normals is rather awful: just perturbing it by Perlin noise does not give a very uniform distribution of directions, some Halton or Sobel sequence would probably be better but this will do for now. Also not we add a small value too our noise sampling points to avoid the zeo knots in the Perlin noise.

Also note that I got crashes with Blender's daily build version before March 4 (win-64 version) when using the getmessage() function. This appears to be fixed in the latest builds but be aware.

Example node setup

Note that the node does not take a position or normal as input, it gets those from the global P and N respectively but of course if necessary this can be changed quite easily. The setup shown below is the exact setup used for the example images at the start of the page.



If you would like to know more about programming OSL you might be interested in my book "Open Shading Language for Blender". More on the availability of this book and a sample can be found on this page.