Element silhouette outline shader

[Dzsozi (h03)]

Hello!

I would like to make a shader which creates an outline for an element I choose to. Similar to this effect on the image (look at the right bottom corner and the desk, you can see a blue and a yellow outline for the objects):

One more example:

I have a .fx file and a few lines of code, but I am not sure what should I do with the shader file, I just started learning HLSL language and I almost know nothing about it. I found this shader on a tutorial website for HLSL, here: http://rbwhitaker.wikidot.com/toon-shader and I just copied the parts that I think I need, the outline part of the shader. But my script doesn't seem to work, I don't know what could be the problem and how to solve it, and also, debugscript 3 doesn't output any errors. It would be awesome if somebody could help me out with this one.

Here's my current client side script:

local outlineShader

function createElementOutline(element, thickness, r,g,b,a)
	if element and isElement(element) then
		for _,name in ipairs(engineGetModelTextureNames(tostring(getElementModel(element)))) do
			dxSetShaderValue(outlineShader, "Texture", name)
			engineApplyShaderToWorldTexture(outlineShader, name, element)
		end
		
		dxSetShaderValue(outlineShader, "LineThickness", thickness)
		dxSetShaderValue(outlineShader, "LineColor", r/255, g/255, b/255, a/255)
	end
end

addEventHandler("onClientResourceStart", resourceRoot,
function()
    if getVersion().sortable < "1.5" then
        outputChatBox("Resource is not compatible with this client.")
        return
	else
		outlineShader = dxCreateShader("shaders/outline.fx")
		
		createElementOutline(localPlayer, 1, 255,255,255,255)
		
		if (not outlineShader) then
            outputChatBox("Could not create Outline shader. Please use debugscript 3.")
        end
    end
end)

And the shader file:

// The world transformation
float4x4 World;
 
// The view transformation
float4x4 View;
 
// The projection transformation
float4x4 Projection;
 
// The transpose of the inverse of the world transformation,
// used for transforming the vertex's normal
float4x4 WorldInverseTranspose;


// The color to draw the lines in.  Black is a good default.
float4 LineColor = float4(0, 0, 0, 1);
 
// The thickness of the lines.  This may need to change, depending on the scale of
// the objects you are drawing.
float LineThickness = .03;


// The texture being used for the object
texture Texture;
 
// The texture sampler, which will get the texture color
sampler2D textureSampler = sampler_state 
{
    Texture = (Texture);
    MinFilter = Linear;
    MagFilter = Linear;
    AddressU = Clamp;
    AddressV = Clamp;
};


struct AppToVertex
{
    float4 Position : POSITION0;            // The position of the vertex
    float3 Normal : NORMAL0;                // The vertex's normal
    float2 TextureCoordinate : TEXCOORD0;    // The texture coordinate of the vertex
};

// The structure used to store information between the vertex shader and the
// pixel shader
struct VertexToPixel
{
    float4 Position : POSITION0;
    float2 TextureCoordinate : TEXCOORD0;
    float3 Normal : TEXCOORD1;
};

// The vertex shader that does the outlines
VertexToPixel OutlineVertexShader(AppToVertex input)
{
    VertexToPixel output = (VertexToPixel)0;
 
    // Calculate where the vertex ought to be.  This line is equivalent
    // to the transformations in the CelVertexShader.
    float4 original = mul(mul(mul(input.Position, World), View), Projection);
 
    // Calculates the normal of the vertex like it ought to be.
    float4 normal = mul(mul(mul(input.Normal, World), View), Projection);
 
    // Take the correct "original" location and translate the vertex a little
    // bit in the direction of the normal to draw a slightly expanded object.
    // Later, we will draw over most of this with the right color, except the expanded
    // part, which will leave the outline that we want.
    output.Position    = original + (mul(LineThickness, normal));
 
    return output;
}
 
// The pixel shader for the outline.  It is pretty simple:  draw everything with the
// correct line color.
float4 OutlinePixelShader(VertexToPixel input) : COLOR0
{
    return LineColor;
}
 
// The entire technique for doing toon shading
technique Toon
{
    // The first pass will go through and draw the back-facing triangles with the outline shader,
    // which will draw a slightly larger version of the model with the outline color.  Later, the
    // model will get drawn normally, and draw over the top most of this, leaving only an outline.
    pass Pass1
    {
        VertexShader = compile vs_2_0 OutlineVertexShader();
        PixelShader = compile ps_2_0 OutlinePixelShader();
        CullMode = CW;
    }
}

I guess I will need to use mta-helper.fx to get some positions, maybe the camera position, I am not sure, I don't really understand yet how shaders work.

Thank you for your reply in advance!

Edited January 31, 2018 by Dzsozi (h03)

[quindo]

I suggest looking into this shader: https://community.multitheftauto.com/?p=resources&s=details&id=7615
you would need to make depth buffer checks to make it not show through walls tho i think.
I use a bit modified version of this shader in my project, and it seems to work pretty much like what you want: image

[Dzsozi (h03)]

I know about this resource and I get it, but

5 minutes ago, quindo said:

you would need to make depth buffer checks to make it not show through walls

|
|
V

 

47 minutes ago, Dzsozi (h03) said:

I am not sure what should I do with the shader file, I just started learning HLSL language and I almost know nothing about it

And I guess I would have to edit the .fx file, but I don't know what to do, edit where and what? I have no idea.

[Slim]

https://wiki.multitheftauto.com/wiki/File:PedShellScreen.jpg

This might work better for you, I haven't tested it but it does have an option to modify the color of the shell.

[Dzsozi (h03)]

This is not an outline, just scaled up textures which is not the same. I already tested it.

[Slim]

On 2/2/2018 at 08:36, Dzsozi (h03) said:

This is not an outline, just scaled up textures which is not the same. I already tested it.

Well then maybe you should figure it out yourself.

[Dzsozi (h03)]

3 hours ago, Justin|X5| said:

Well then maybe you should figure it out yourself.

I wouldn't have opened a topic here if I could do it by myself. That doesn't help me out much.

[znajder]

I write to you small stuff looks littlebit like some shiiit, but this is maximum what we can get from GTA Engine (i think).

 

Code:

-- created by l0nger 

local outlineForPlayers={}
local DEFAULT_DRAW_DISTANCE = 50
local DEFAULT_COLOR = {255, 200, 100, 125}
local DEFAULT_SPECULAR_POWER = 1.15

function createOutlineForPlayer(player)
  if outlineForPlayers[player] then return false end
  outlineForPlayers[player]={}

  outlineForPlayers[player].shader=dxCreateShader("fx/outline.fx", 1, DEFAULT_DRAW_DISTANCE, true, "ped")
  if not outlineForPlayers[player].shader then
    -- we cant make shader, problem with memory?
    return false
  end

  dxSetShaderValue(outlineForPlayers[player].shader, "sColorizePed", {DEFAULT_COLOR[1]/255, DEFAULT_COLOR[2]/255, DEFAULT_COLOR[3]/255, DEFAULT_COLOR[4]/255})
  dxSetShaderValue(outlineForPlayers[player].shader, "sSpecularPower", DEFAULT_SPECULAR_POWER)

  engineApplyShaderToWorldTexture(outlineForPlayers[player].shader, "*", player)
  engineRemoveShaderToWorldTexture(outlineForPlayers[player].shader, "muzzle_texture*", player)

  if getElementAlpha(player)==255 then
    setElementAlpha(player, 254)
  end
  return true
end

function removeOutlineFromPlayer(player)
  if not outlineForPlayers[player] then return false end
  if not isElement(outlineForPlayers[player].shader) then return false end

  engineRemoveShaderToWorldTexture(outlineForPlayers[player].shader, "*", player)
  destroyElement(outlineForPlayers[player].shader)
  outlineForPlayers[player] = nil
  return true
end

-- and creating for local player
createOutlineForPlayer(localPlayer)

 

And code from outlife.fx

https://pastebin.com/r2rD9ss4

 

Regards and hf.  

Edited February 6, 2018 by l0nger

[Dzsozi (h03)]

Thank you for your reply, but this is still not the effect I would like to reach. There's the cartoon shader, it has an outline shader that is being applied for everything, I just need to know how can I get this outline and apply it to an element I choose.

[Ren_712]

On 31.01.2018 at 17:52, Dzsozi (h03) said:

..And I guess I would have to edit the .fx file, but I don't know what to do, edit where and what? I have no idea.

set render state ZEnable = true; to enable depth testing (you'll find it in line 71 in effect file) ped_wall_mrt.fx (allso in the fallback effect ped_wall.fx)

Edited February 27, 2018 by Ren_712