Creating Materials and Lights in the Visual Layer

In today’s post, we’re going wrap up this series by combining everything we’ve learned so far and take you through the steps in creating a custom material. We also have an amazing new tool to show you that empowers anyone to design a custom material. To see how you can use these custom materials in your XAML app, be sure to check out the last two posts in this series; XAML and Visual Layer Interop, part one and part two.

The Fluent Design Language is an evolving concept, rather than a one-time design language release like MDL and MDL2. It was designed to expand and grow as Microsoft and the community of creators (developers and designers), adds to what it could be. We’ll show you that anything is possible, as designers, developers and the Windows community have common ground to share their creations and create amazing materials.

Let’s get started by first showing you how a material is created by chaining effects, then we’ll explore using the new Material Creator to easily and quickly create materials.

Creating Material with the Visual Layer

CompositionEffectBrush

Whether we’ll be using the effect in a XamlCompositionBrushBase or painting a SpriteVisual, the construction of the effect graph is the core of the approach. In order to create the material we want, we’ll need the following components:

• Effect sources: A SurfaceBrush for the NormalMap and a BackdropBrush for access to the pixels underneath the material
• The effect graph: A composite of different effects to control the Material’s reflectance properties and filter effects (such as blur and tint) to customize for UI usage
• Lighting: The Visual is in a scene that has a CompositionLight applied

Let’s start with the first source, a SurfaceBrush. This will be provided by using LoadedImageSurface to load a NormalMap.

NormalMap and LoadedImageSurface

If you’ve had any experience with 3D computer graphics, maybe as a game developer, you may already know what a normal map is and the image above looks familiar. If you’ve never worked with one before, Normal mapping is a technique that determines the reflectance of light at every pixel (read more about Normal mapping here). The Visual Layer in Windows 10 gives you a choice of industry standard reflectance models; Blinn-Phong and Physically Based Blinn-Phong. Go here to read more about the math behind how this is done.

For today’s demo, we used a 2D picture of a textured surface and transformed it into a Normal map image using an image editor. There are many image editing tools that let you do this. You can use any one you prefer to create your image.

To get started, we can load the image using the new LoadedImageSurface API. Let’s add a private field for the LoadedImageSurface, load the image and create a CompositionSurfaceBrush with it.

// Load NormalMap onto an ICompositionSurface using LoadedImageSurface
LoadedImageSurface _surface = LoadedImageSurface.StartLoadFromUri(new Uri("ms-appx:///Images/NormalMap.jpg"), new Size(512,384));

// Load Surface onto SurfaceBrush
CompositionSurfaceBrush normalMap = compositor.CreateSurfaceBrush(_surface);
normalMap.Stretch = CompositionStretch.Uniform;

Now we’re ready to move on to creating and chaining effects.

Chaining Effects to create the effect graph

For our material, we are going to create a chain of effects that leverages the Win2D’s ArithmeticCompositeEffect (note: be sure to add the Win2D NuGet package to your project). All effects can be used as input sources for other effects, thus enabling you to allow a chain of effects to one or more inputs.

ArithmeticCompositeEffect lets you assign two sources for the composite, giving each one a weight toward the final effect. For example, Source1 at 0.75 (75%) and Source2 at 0.25 (25%). You can also use an additional ArithmeticCompositeEffect as one of the sources to add more effects in the composite chain.

Let’s step back for a minute and think about how we want create the composite:

• Parent ArithmeticCompositeEffect to be used for Brush
  • Source 1: Child ArithmeticCompositeEffect
    • Source 1: ColorSourceEffect for tint coloring
    • Source 2: GaussianBlurEffect using the BackDropBrush for its source
  • Source2: SceneLightingEffect using the Normal map for its source

For source 1, we’ll combine a ColorSourceEffect and GaussianBlurEffect (from Win2D) with a nested ArithmeticSourceEffect. For Source 2, we’ll use a SceneLightingEffect (from Windows.UI.Composition.Effects). This will manipulate the reflective properties of the effect’s source when a CompositionLight, from a XamlLight for example, is applied.

Note that the SceneLightingEffect is used to modify the default lighting applied to the contents of a SpriteVisual targeted by a CompositionLight. In today’s example, we are going to create SurfaceBrush using a NormalMap (loaded by LoadedImageSurface) to define dents and bumps that the light reflects off of.

Furthermore, in order to use the SceneLightingEffect, the content being modified must be defined as one of the sources into a multi-input effect graph, with the other input being the SceneLightingEffect. For example, above, the content whose lighting properties are being modified is defined by Source1 of the parent ArithmeticCompositeEffect.

Here’s what the code looks like for the effect graph:

// Define Effect graph
const float glassLightAmount = 0.5f;
const float glassBlurAmount = 0.95f;
Color tintColor = Color.FromArgb(255, 128, 128, 128);

var graphicsEffect = new ArithmeticCompositeEffect
{
    Name = "LightComposite",
    Source1Amount = 1,
    Source2Amount = glassLightAmount,
    MultiplyAmount = 0,
    // Nested Composite to combine the Blur and Color tint effects
    Source1 = new ArithmeticCompositeEffect
    {
        Name = "BlurComposite",
        Source1Amount = 1 - glassBlurAmount,
        Source2Amount = glassBlurAmount,
        MultiplyAmount = 0,
        Source1 = new ColorSourceEffect
        {
            Name = "Tint",
            Color = tintColor,
        },
        Source2 = new GaussianBlurEffect
         {
            BlurAmount = 20,
             Source = new CompositionEffectSourceParameter("Backdrop"),
            Optimization = EffectOptimization.Balanced,
            BorderMode = EffectBorderMode.Hard,
        },
    },
    // The SceneLighting effect, which will use a NormalMap
    Source2 = new SceneLightingEffect
    {
        AmbientAmount = 0.15f,
        DiffuseAmount = 1,
        SpecularAmount = 0.1f,
        NormalMapSource = new CompositionEffectSourceParameter("NormalMap")
    }
};

Notice the SceneLightingEffect’s NormalMapSource property and the GaussianBlurEffect’s Source; these are parameter provided sources. We will set what these parameters are as we pull everything together to create the CompositionEffectBrush:

// Create EffectFactory and the CompositionEffectBrush
CompositionEffectFactory effectFactory = Window.Current.Compositor.CreateEffectFactory(graphicsEffect);
CompositionEffectBrush effectBrush = effectFactory.CreateBrush();

// Create BackdropBrush, this is used by the GaussianBlurEffect
CompositionBackdropBrush backdrop = Window.Current.Compositor.CreateBackdropBrush();

// Set Sources to Effect
effectBrush.SetSourceParameter("NormalMap", _normalMap);
effectBrush.SetSourceParameter("Backdrop", backdrop);

With the CompositionEffect completed, we can now use it to paint a SpriteVisual, like this:

SpriteVisual spriteVisual = Window.Current.Compositor.CreateSpriteVisual();
spriteVisual.Brush = effectBrush;

If you’re primarily a XAML dev, you can use this effect in a XamlCompositionBrushBase. Let’s take a look.

Using the CompositionEffectBrush in a XamlCompositionBrushBase

As I mentioned earlier, we can also create this effect graph in XamlCompositionBrushBase and set the XamlCompositionBrushBase’s CompsositionBrush property. If you haven’t read the post in this series on how to create a XamlCompositionBrushBase, go here to catch up.

As with the other XamlCompositionBrushBase implementations, we build the effect graph in the OnConnected method and make sure that the user’s device supports effects. This can be done using the AreEffectsSupported method of the CompositionCapabilities API.

Here’s the full class:

public sealed class MaterialBrush : XamlCompositionBrushBase
{
    private LoadedImageSurface _surface;

    protected override void OnConnected()
    {
        if (DesignMode.DesignModeEnabled) return;

        Compositor compositor = Window.Current.Compositor;

        // CompositionCapabilities: Are Effects supported?
        bool usingFallback = !CompositionCapabilities.GetForCurrentView().AreEffectsSupported();
        FallbackColor = Color.FromArgb(100, 60, 60, 60);

        if (usingFallback)
        {
            // If Effects are not supported, use Fallback Solid Color
            CompositionBrush = compositor.CreateColorBrush(FallbackColor);
            return;
        }

        // Load NormalMap onto an ICompositionSurface using LoadedImageSurface
        _surface = LoadedImageSurface.StartLoadFromUri(new Uri("ms-appx:///Images/NormalMap.jpg"), new Size(512, 384));

        // Load Surface onto SurfaceBrush
        CompositionSurfaceBrush normalMap = compositor.CreateSurfaceBrush(_surface);
        normalMap.Stretch = CompositionStretch.Uniform;

        // Define Effect graph
        const float glassLightAmount = 0.5f;
        const float glassBlurAmount = 0.95f;
        Color tintColor = Color.FromArgb(255, 128, 128, 128);

        var graphicsEffect = new ArithmeticCompositeEffect()
        {
            Name = "LightComposite",
            Source1Amount = 1,
            Source2Amount = glassLightAmount,
            MultiplyAmount = 0,
            Source1 = new ArithmeticCompositeEffect()
            {
                Name = "BlurComposite",
                Source1Amount = 1 - glassBlurAmount,
                Source2Amount = glassBlurAmount,
                MultiplyAmount = 0,
                Source1 = new ColorSourceEffect()
                {
                    Name = "Tint",
                    Color = tintColor,
                },
                Source2 = new GaussianBlurEffect()
                {
                    BlurAmount = 20,
                    Source = new CompositionEffectSourceParameter("Backdrop"),
                    Optimization = EffectOptimization.Balanced,
                    BorderMode = EffectBorderMode.Hard,
                },
            },
            Source2 = new SceneLightingEffect()
            {
                AmbientAmount = 0.15f,
                DiffuseAmount = 1,
                SpecularAmount = 0.1f,
                NormalMapSource = new CompositionEffectSourceParameter("NormalMap")
            },
        };

        // Create EffectFactory and EffectBrush
        CompositionEffectFactory effectFactory = compositor.CreateEffectFactory(graphicsEffect);
        CompositionEffectBrush effectBrush = effectFactory.CreateBrush();

        // Create BackdropBrush
        CompositionBackdropBrush backdrop = compositor.CreateBackdropBrush();

        // Set Sources to Effect
        effectBrush.SetSourceParameter("NormalMap", normalMap);
        effectBrush.SetSourceParameter("Backdrop", backdrop);

        // Set EffectBrush as the brush that XamlCompBrushBase paints onto Xaml UIElement
        CompositionBrush = effectBrush;
    }

    protected override void OnDisconnected()
    {
        // Clean up resources
        _surface?.Dispose();
        _surface = null;

        CompositionBrush?.Dispose();
        CompositionBrush = null;
    }
}

To see this in action, let’s create a Grid to put in the middle of our page’s root Grid and set an instance of our MaterialBrush to that Grid’s Background brush:

<Grid Background="Gray">
    <Grid Width="580"
          Height="387"
          HorizontalAlignment="Center"
          VerticalAlignment="Center">

        <!-- Our new MaterialBrush -->
        <Grid.Background>
            <brushes:MaterialBrush />
        </Grid.Background>
    </Grid>
</Grid>

Here’s what it would look like if you ran the app now:

This is because you’re missing the second part of the approach, the lighting!

Illuminating the Material with Lights

In the last post, we created two lights (an AmbientLight “AmbLight” and the SpotLight “HoverLight”). We’ll use them today to apply lighting to the UIElement that is using our custom material.

Since our MaterialBrush uses the new SceneLightingEffect with a Normal map, any lights applied will enhance the material per the SceneLightingEffect’s configuration. Note that this isn’t necessary, but can greatly enhance your material. For example, if you’re using an Acrylic material in your app, adding Reveal will enhance the Acrylic.

Let’s now add the two XamlLights to the Grid:

<Grid Background="Gray">
    <Grid Width="580"
          Height="387"
          HorizontalAlignment="Center"
          VerticalAlignment="Center">
        <Grid.Background>
            <brushes:MaterialBrush />
        </Grid.Background>

        <!-- Added lights -->
        <Grid.Lights>
            <lights:HoverLight />
            <lights:AmbLight />
        </Grid.Lights>
    </Grid>
</Grid>

Now, this is what you’ll see at runtime:

What if it were easier to create and experiment with new materials? What if there were a tool that anyone can use? Let’s take a look at what’s coming to the WindowsUIDevLabs GitHub repo, the Material Creator tool.

Using the new Material Creator

Introducing availability of the new Material Creator tool!

Creating custom materials may sometimes requires a bit of experimentation and tweaking to get the effect’s property configuration just right. This would take time if you had to constantly tweak and redeploy your app. What if there were a way that you could change effect properties and material layers in real time?

The Material Creator can be found on the WindowsUIDevLabs GitHub repo in the demos folder here. (Note: you need to be running Windows 10, build 16225 or higher to use the Material Creator).

Generating the SceneLightingEffect code

One of the great features of the tool is being able to see the effect graph after you’re done creating the material. Click the ellipsis next to the save button and select “view effect graph” to see the C# code for the SceneLightingEffect. You can then copy and use this code directly in your custom material class.

If you go back up to the part of this article where we created the ArithemticCompositeEffect that contains a SceneLightingEffect, that’s where you can use this code!

Saving and Loading Materials

The Material Editor can also save and load materials! If you want to save your current progress on a material, or share a completed material with another developer, just click the Save button and it will create a json file containing all the layers and effect configurations. To load an existing material or edit a material shared with you, just click Load and select the json file.

The key takeaway is that you don’t need to be a developer to create materials. A designer can create a material and then share the saved json file with a developer for implementation in a XamlCompositionBrushBase. Even Windows enthusiasts, like the Windows Insiders, can start building out a universe of materials to drive the evolution of Fluent Design.

Blog Series Wrap up: The future of Fluent Design materials

Acrylic and Reveal are stunning examples of how using Material with Lights can alter the Windows experience, but they’re just the beginning of what is possible. The vision for the Fluent Design Language going forward is that developers and designers can easily build custom materials, innovate and share as well.

The message we want you to walk away with is that you can build new Materials, for a couple primary reasons:

• Because you’re a Creator
• Because it embodies your brand

We look forward to seeing what kinds of materials you create for your Windows apps! If you’ve already built your own material, feel free to share in the comments below.

Resources

• Demo Code:
  • Material Creator
  • Brush Interop
  • Light Interop
• Blog posts: XAML and Visual Layer Interop series
  • Part One
  • Part Two
• Lighting Overview
• CompositionCapabilities API
• Light Types
• Mathematics of lighting
• SceneLightingEffect
• WindowsUIDevLabs GitHub repo (contains Samples Gallery app)
• Win2D Effects API Documentation

The post Creating Materials and Lights in the Visual Layer appeared first on Building Apps for Windows.