Port of IPToCountry and some questions to port from ES To SP

[decompile]

Hello again!

I tried to port the ES IPToCountry to SP but it seems like it does not include StringIO, so could someone might port that for us to SP please? (Never used it so i have no clear about that)

Questions:

1. Whats the function for sending a userid a message to CHAT with colors like \x07..? (ES: esc.tell)

2. Whats the function for sending a message to all users to CHAT with colors like \x07..? (ES: esc.msg)

3. Whats the replace for es.getUseridList()?

4. Is there something for es.registerClientCommandFilter ?

5. How can I execute something through the server ? es.server.queuecmd

And last question:

Can someone might explain me why we are using indexes from players instead of the userid like on es (If I understood that correctly)


That would be the questions for porting my first "easier" files to SP.

Thank you!

[satoon101]

For 1 and 2, use messages.SayText2. There are lots of examples on the forums already.

For player iteration, use filters.players.PlayerIter.

To register a client command filter, use commands.client.ClientCommandFilter. If you wish to only listen for a particular command or a few particular commands, you should use commands.client.ClientCommand instead.

To execute a command, use engines.server.engine_server.server_command (sorry, it isn't on the wiki, yet).

Syntax: Select all

from engines.server import engine_server

engine_server.server_command('changelevel de_dust2;')

Note that currently you must end the server_command string with either ; or \n. We are going to add that internally here soon, so that it won't be required anymore.

Also, if you need the command to be executed immediately, you can use engine_server.server_execute() to force all currently queued commands to execute.

You can still use userids instead of indexes for your own purposes and probably should in some cases. The main reason we are using indexes for a lot of things is that our Entity implementation uses them. The Player class inherits from Entity, and as such uses the index, as well.

*Edit: also, we are working very close to the engine, and pretty much nothing in the engine uses userids, only indexes (and other object types like PlayerInfo/edict_t).

[decompile]

Thank you!

[decompile]

Short question towards IPToCountry SP port..

I have tried to port

Syntax: Select all

def _download_database(self, url):
  try:
     with closing(urllib.urlopen(url)) as u:
        return StringIO(u.read())
  except IOError:
     self.__show_exception(sys.exc_info())
  return None

to

Syntax: Select all

def _download_database(self, url):
  try:
     with closing(urllib.request.urlopen(url)) as u:
        response = u.read().decode('utf-8')
        return StringIO(response)
  except IOError:
     self.__show_exception(sys.exc_info())
  return None

But im still getting

utf-8 codec can't decode byte 0x8f in position 12: invalid start byte

It tells me that it needs to be Str or None, not bytes but seems like decode doesnt work for that

I tried googling it and found actually almost always the same answer...

[L'In20Cible]

Please, post full code to reproduce and full traceback.

[iPlayer]

You sure you're getting UTF-8 from u.read()?

I can assume that you're actually getting binary data from that, and StringIO won't work in this case.
Try adapting your code to ByteIO.

[Doldol]

Why don't you just use a readily-available geoip library? e.g. https://pypi.python.org/pypi/geoip2

[decompile]

Why don't you just use a readily-available geoip library? e.g. https://pypi.python.org/pypi/geoip2

I could do that too, but i see someone already ported it over to es and porting it to SP shouldnt be that hard at all since im already used to that for years now.

[satoon101]

Could you please do this so we can better assist you:

Please, post full code to reproduce and full traceback.

[satoon101]

Actually, had a bit of time to do some testing this morning. Haven't figured out myself, but here is full reproducible code:

Syntax: Select all

>>> from contextlib import closing
>>> from urllib.request import urlopen
>>> from io import StringIO
>>> with closing(urlopen('http://software77.net/geo-ip?DL=2')) as u:
	x = StringIO(u.read())

	
Traceback (most recent call last):
  File "<pyshell#5>", line 2, in <module>
    x = StringIO(u.read())
TypeError: initial_value must be str or None, not bytes
>>> with closing(urlopen('http://software77.net/geo-ip?DL=2')) as u:
	response = u.read().decode('utf-8')

	
Traceback (most recent call last):
  File "<pyshell#7>", line 2, in <module>
    response = u.read().decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x94 in position 12: invalid start byte

[Doldol]

Actually, had a bit of time to do some testing this morning. Haven't figured out myself, but here is full reproducible code:

You'll have to use BytesIO instead of StringIO, since you're downloading binary data.

Edit: But that doesn't make sense as u.read() should already return a bytes object, I tried the code out and I think no one noticed you're downloading a zip archive, which obviously can't be handled by StringIO.

[satoon101]

The base of this code comes directly from SuperDave's IP to Country addon for ES. That is where the url I added comes from. As far as I know, it still works fine on ES using Python2.5. It then takes the StringIO object and parses the zip from that using zipfile.ZipFile.

*Edit: also, using this code in Python2.7 works fine (obviously with urlopen and StringIO imports being their 2.7 versions).

[iPlayer]

Because in Python 2 str can be used for both binary and text data, in Python 3 it can't.

[Doldol]

Working example:

Syntax: Select all

from pprint import pprint
from zipfile import ZipFile
from contextlib import closing
from urllib.request import urlopen
from io import StringIO, BytesIO
import csv

with closing(urlopen('http://software77.net/geo-ip?DL=2')) as u:
    data = u.read()

zip_file = ZipFile(BytesIO(data))
csv_file = zip_file.read(zip_file.namelist()[0])
csv_reader = csv.reader(StringIO(csv_file.decode("utf-8", "ignore")))

pprint(list(csv_reader)[:1000])

[Doldol]

Btw, I threw this together kind of quickly, it's a wrapper around geoip2, it downloads and keeps the database updated (uses threading to do this) & contains an example.

The attachment geoip2sp.zip is no longer available

[decompile]

Btw, I threw this together kind of quickly, it's a wrapper around geoip2, it downloads and keeps the database updated (uses threading to do this) & contains an example.

Thank you! Im gonna try that tomorrow After work. Thank you for everyone who replied.

[decompile]

Okay, i tried it out now.. It seems to work except it only shows "None" for almost every player who connects.. Always getting

geoip2.errors.AddressNotFoundError: The address XXX.XXX.XXX.XX is not in the dat
abase.

[satoon101]

Using the old IP-to-Country way, this seems to work just fine:

Syntax: Select all

>>> from zipfile import ZipFile
>>> from contextlib import closing
>>> from urllib.request import urlopen
>>> from io import BytesIO
>>> from io import StringIO
>>> with closing(urlopen('http://software77.net/geo-ip?DL=2')) as u:
	data = u.read()

	
>>> z = ZipFile(BytesIO(data))
>>> csv = z.read(z.namelist()[0])
>>> csv = StringIO(csv.decode('utf-8', 'ignore'))
>>> database = list()
>>> for line in csv.readlines():
	if line.startswith('#'):
		continue
	if line.count(',') < 6:
		continue
	values = line.strip().replace(r'"', '').split(',')
	database.append({'ip_to': float(values[1]), 'country_3': values[5].strip(), 'country_long': values[6]})

	
>>> def get_country(ip):
	new = ip.split(':', 1)[0]
	ip_list = [int(x) for x in ip.split('.')]
	if len(ip_list) != 4:
		raise ValueError('Invalid IP address "{0}".'.format(ip))
	long_ip = ip_list[3] + (ip_list[2] << 8) + (ip_list[1] << 16) + (ip_list[0] << 24)
	for data in database:
		if long_ip <= data['ip_to']:
			return data['country_long'], data['country_3']

		
>>> l = get_country('123.123.123.123')
>>> print(l)
('China', 'CHN')

I really don't know how often you would ideally want to update the database, but I would guess not very often would it really be necessary.

[decompile]

Should i just use that as a new package or just update it into the "iptocountry" file?

[Painkiller]

Using the old IP-to-Country way, this seems to work just fine:

Syntax: Select all

>>> from zipfile import ZipFile
>>> from contextlib import closing
>>> from urllib.request import urlopen
>>> from io import BytesIO
>>> from io import StringIO
>>> with closing(urlopen('http://software77.net/geo-ip?DL=2')) as u:
	data = u.read()

	
>>> z = ZipFile(BytesIO(data))
>>> csv = z.read(z.namelist()[0])
>>> csv = StringIO(csv.decode('utf-8', 'ignore'))
>>> database = list()
>>> for line in csv.readlines():
	if line.startswith('#'):
		continue
	if line.count(',') < 6:
		continue
	values = line.strip().replace(r'"', '').split(',')
	database.append({'ip_to': float(values[1]), 'country_3': values[5].strip(), 'country_long': values[6]})

	
>>> def get_country(ip):
	new = ip.split(':', 1)[0]
	ip_list = [int(x) for x in ip.split('.')]
	if len(ip_list) != 4:
		raise ValueError('Invalid IP address "{0}".'.format(ip))
	long_ip = ip_list[3] + (ip_list[2] << 8) + (ip_list[1] << 16) + (ip_list[0] << 24)
	for data in database:
		if long_ip <= data['ip_to']:
			return data['country_long'], data['country_3']

		
>>> l = get_country('123.123.123.123')
>>> print(l)
('China', 'CHN')

I really don't know how often you would ideally want to update the database, but I would guess not very often would it really be necessary.

I have tested this but it did not come to the desired display.
Example what it should show: