SourceMod interop

[fakuivan]

I've been developing SourceMod plugins and extensions for a long time now and I feel quite comfortable digging into the source and reading documentation, but one thing I can't stand is SourcePawn, the mediocre language SourceMod plugins are coded in. Python on the other hand is a wonderful language, one of my favorites, I think that most people feel the same too, given that is one of the most used languages out there.

I feel like some sort of bridge intercom between Source.Python plugins and SourceMod plugins would be highly beneficial to both projects. Given that Python can be slow sometimes (and this is especially sensitive in per-frame tasks), but it can be a joy to write, while in the other hand, SourcePawn is quite fast, but it can be very frustrating to code and maintain projects of a decent size.

Technically, SourceMod provides a "native" interface that allows plugins to call functions declared by extensions or other plugins, that along with a forward api should be enough to make something like this work on the SourceMod side, without any modifications to the mod itself.
Sadly I don't have much knowledge on Source.Python to know how to tackle this problem from that side. I'd appreciate some advice on that.

Thanks for reading!

[Ayuto]

I just had a look at it and it seems doable. Here is my proof of concept for creating natives. It's using AddNatives to add natives created with SP to the core natives of SM. In a real bridge (not proof of concept) we wouldn't use that function, but create an SM extension instead that exposes a C function that accept a name (name of the native) and a function pointer (the native itself). It then adds the natives to its extension natives instead of adding it to the core natives. We should also create a Python extension module that exposes some parts of the SourcePawn language. In this proof of concept, all of this is done via the memory module, which looks more difficult than it is.

1. Create ../sourcemod/scripting/include/sp.inc. This defines the native we will create with SP:

Code: Select all

native int my_multiply(int x, int y);


2. Create ../sourcemod/scripting/sp.sp. This is our SM plugin to test the native created with SP:

Code: Select all

#include <sp>

public void OnPluginStart() {
    PrintToServer("Result: %i", my_multiply(3, 7));
}

3. Compile the plugin and copy sp.smx file to ../sourcemod/plugins.
4. Create the SP plugin (../source-python/plugins/sm/sm.py) that adds the native to SM:

Syntax: Select all

from paths import GAME_PATH

import memory
from memory import Callback
from memory import Convention
from memory import DataType
from memory import NULL

LOGIC_PATH = GAME_PATH / 'addons/sourcemod/bin/sourcemod.logic.so'
if not LOGIC_PATH.isfile():
    raise ValueError(f'{LOGIC_PATH} does not exist. This POC only works on Linux.')
    
binary = memory.find_binary(LOGIC_PATH, check_extension=False)

# void AddNatives(sp_nativeinfo_t *natives)
AddNatives = binary['_ZL10AddNativesP15sp_nativeinfo_s'].make_function(
    Convention.CDECL,
    [DataType.POINTER],
    DataType.VOID)

# typedef cell_t (*SPVM_NATIVE_FUNC)(SourcePawn::IPluginContext *, const cell_t *);
@Callback(
    Convention.CDECL,
    [DataType.POINTER, DataType.POINTER],
    DataType.INT)
def multiply(args):
    print('Native called')
    context = args[0]
    params = args[1]
    
    print('Param count:', params.get_int())
    return params.get_int(4) * params.get_int(8)

# Construct the natives array
"""
typedef struct sp_nativeinfo_s
{
	const char		*name;	/**< Name of the native */
	SPVM_NATIVE_FUNC func;	/**< Address of native implementation */
} sp_nativeinfo_t;
"""
native_name = 'my_multiply'

natives = memory.alloc(16)

# First native
natives.set_string_pointer(native_name)
natives.set_pointer(multiply, 4)

# Second/null native to tell SM this is the end of the array
natives.set_pointer(NULL, 8)
natives.set_pointer(NULL, 12)

# Add the native(s)
AddNatives(natives)

Now, it's time to test:

Code: Select all

sp plugin load sm
[SP] Loading plugin 'sm'...
[SP] Successfully loaded plugin 'sm'.
sm plugins load sp
Native called
Param count: 2
Result: 21
[SM] Loaded plugin sp.smx successfully.


[fakuivan]

Thanks for that amazing response! I am currently working on handling dynamically assigned natives, using libffcall (https://github.com/fakuivan/SMConnect/b ... pp#L38-L50)

I see that it is possible to use ``memory`` module to call exported functions on extensions. Now how would we go about handling sp loading first and then trying to query a sourcemod extension but failing because of it not being loaded?

SP loads when the SM extension is ready to process requests (loads late):
[*] SP's entry point -> Third party SourceMod extension that exposes symbols
SM ext loads when SP is ready to process requests (loads earlier):
[*] SM ext entry point -> ?

[Ayuto]

I see that it is possible to use ``memory`` module to call exported functions on extensions.

AddNatives is not an exported function. The memory module uses its private symbol to get its address. The address is then called by the memory module. So, the memory module goes deeper than just calling a public/exported function.

Now how would we go about handling sp loading first and then trying to query a sourcemod extension but failing because of it not being loaded?

So, you basically want to know of to check with SP whether an SM extension has been loaded?

Btw. you should always load SP after SM. Otherwise SM might not be able to find all function signatures (e.g. if a function has been hooked with SP). SP does find them, because it also looks for hooked functions. So, you actually don't need an answer for that question (in this case).

[fakuivan]

Great news. So SP should always load after SM, are there any mechanisms in place that detect mms/sm and do this automatically or is it up to the user?

[Ayuto]

It's up to the user by loading the plugins via autoexec.cfg.

[Zeus]

I'm a bit confused; does this mean we can receive forwards from sourcemod plugins?

[Ayuto]

The example above creates a native with Source.Python and makes it available in Sourcemod, so SM plugins can call an SP function. I'm pretty sure we can also make it work vice-versa (call SM natives from SP) or even receive forwards.

[fakuivan]

I'm a bit confused; does this mean we can receive forwards from sourcemod plugins?

That is not the point of the example, but it can be done easily with an extension that can already communicate with a source.python script.

[Zeus]

The example above creates a native with Source.Python and makes it available in Sourcemod, so SM plugins can call an SP function. I'm pretty sure we can also make it work vice-versa (call SM natives from SP) or even receive forwards.

Oh, i'd be very interested in that; theres a few plugins i'd like to write that would require forwards from SM. I'm not going to pretend that I understand the memory stuff since i literally stumbled on this project yesterday XD

[Ayuto]

Which forwards exactly do you need? Maybe we already have something similar.

[Zeus]

theres a plugin for TF2 that uploads game logs to a website; in it, it registers 2 forwards:

Code: Select all

   
   // Make it possible for other plugins to get notified when a log has been uploaded
   g_hLogUploaded = CreateGlobalForward("LogUploaded", ET_Ignore, Param_Cell, Param_String, Param_String);
   
   // Let other plugins block log lines
   g_hBlockLogLine = CreateGlobalForward("BlockLogLine", ET_Event, Param_String);
   

http://www.teamfortress.tv/13598/medics ... od-plugin/ (logstf plugin)

[Ayuto]

How does a SourceMod plugin listen to those forwards? (I have almost no knowledge about SM plugins)

[Zeus]

In SM, simply implement the function (with forward on the function stub) and it'll be called when the original plugin makes the forwarding call.

https://wiki.alliedmods.net/Introduction_to_sourcepawn
https://wiki.alliedmods.net/Function_Calling_API_(SourceMod_Scripting)#Global_Forwards

In my case; this happens with this function, which is called on round end:

Code: Select all


CallLogUploaded(bool:success, const String:logid[], const String:url[]) {
    Call_StartForward(g_hLogUploaded);     //https://sm.alliedmods.net/new-api/functions/Call_StartForward

    // Push parameters one at a time
    Call_PushCell(success);
    Call_PushString(logid);
    Call_PushString(url);

    // Finish the call
    Call_Finish();
}


I would just implment this:

Code: Select all


forward void OnLogUploaded(bool success, const char[] logid, const char[] url);
 

SM must keep some kind of reference to all defined forwards?

[Ayuto]

Okay, I came to the conclusion that the easiest way is to create an SM plugin that listens to that forward. Then you can use the code above to call a native created with SP.

So, it basically looks like this:

1. logstf creates a new forward and calls it from time to time.
2. You create a new SM plugin that listens to the forward. The implementation will call a native (e.g. NotifySP_OnLogUploaded(success, logid, url)).
3. You create the native NotifySP_OnLogUploaded with SP using the code above.

Since the forward uses strings you need a little bit more than what I have shown in my first post:

Syntax: Select all

# sp::PluginContext::LocalToString(int, char **)
_LocalToString = None

def get_LocalToString(context):
    global _LocalToString
    if _LocalToString is not None:
        return _LocalToString
        
    return context.make_virtual_function(
        21,
        Convention.THISCALL,
        [DataType.POINTER, DataType.INT, DataType.POINTER],
        DataType.VOID)
        

# typedef cell_t (*SPVM_NATIVE_FUNC)(SourcePawn::IPluginContext *, const cell_t *);
@Callback(
    Convention.CDECL,
    [DataType.POINTER, DataType.POINTER],
    DataType.INT)
def NotifySP_OnLogUploaded(args):
    context = args[0]
    params = args[1]
    buffer = memory.alloc(4)
    LocalToString = get_LocalToString(context)
    
    success = params.get_int(4)
    
    LocalToString(context, params.get_int(8), buffer)
    logid = buffer.get_string_pointer()
    
    LocalToString(context, params.get_int(12), buffer)
    url = buffer.get_string_pointer()
    
    print(f'OnLogUploaded: {x}, {y}')
    
    return 0

[fakuivan]

I have control over an extension binary. How would I go about exposing symbols on the C++ extension so the SP plugin can find them on windows and linux?

[Ayuto]

You only need to declare your functions with extern "C". Then you can retrieve them with the memory module using their actual names. This might help you:
viewtopic.php?f=20&t=1206&p=7848#p7848