Pythomnic3k

• Python version compatibility fixes. Works with 3.3 to 3.8.

• More SSL configuration parameters for all SSL-based protocols.

• Added additional configuration options for resource caching.

• Added support for using msgpack for marshaling RPC calls, presumably more secure than pickle.

• A huge performance optimization (actually modification of the code that was slowing everything down).

• Added optional caching for all resources. This is mostly useful for database resources and such. Cache size, policy and TTL are configurable. Any behaviour more complex than that, such as using custom cache keys, cache content manipulation or synchronization of caches in different processes is possible by specifying custom hook methods.

• Added resource pool meta configuration parameter pool__max_time which restricts request execution time inside a resource on top of current remain of the request timeout. This is useful when you are accessing resources that ought to respond in short time without a possibility of waiting for the entire request timeout.

• Interfaces are restarted automatically when their configuration files change. Similarly if a new interface is added in config_interfaces.py it is started, and stopped if removed. One less reason to ever restart the application.

• Meta configuration file config_config.py contains "macros" expanded in all other configuration files. For example:
config_config.py:

config = dict \
(
ip = "127.0.0.1",
port = "80",
)

config_foo.py:

config = dict \
(
server_address = ("$ip", "eval(int($port))"),
)

• Added custom application module for pre/postprocessing of application modules being loaded. Specifically, when a module foo.py is imported in a

pmnc.foo.bar()

call, framework calls before_reload() and after_reload() from __module_loader__.py This is useful for checksum checking, macro replacements and such.

• Added facility for conditional logging:

if pmnc.log.debug:
    pmnc.log.debug("this is only evaluated in DEBUG")

this helps to avoid evaluation of complex debugging messages.

• New logging level NOISE which is lower than even DEBUG is used by the framework to report boring technical stuff.

• Added automatic fragmentation for short messages sent through an SMPP resource. Supported fragmentation methods include UDH and SAR. Also added 7 bit encoding support.

• Added automatic recovery for the Berkeley DB database used in state facility in case a panic condition event is received.

• Added facility to abort a retried call with failure, not have it retried any more, and not have an error message in log file too.

pmnc.retry.exit()

is similar in behavior to

raise SystemExit()

• Added alternative implementation of the PostgreSQL protocol based on psycopg.

• Added UDP protocol used for sending and receiving separate packets in "one packet = one request" manner.

• Resources of "file" protocol can be used not only for writing files to a shared directory, but also for reading them. This is useful when combined with caching.

• A set of allowed SSL ciphers can be specified in all TCP/SSL based protocols through parameter

ssl_ciphers = "...openssl cipher specification..."

• Python 3.4 compatibility fixes.

• Added dynamic method lookup to modules via __getattr__ method:

__all__ = [ "__getattr__" ]
def __getattr__(name):
    return f

• Added shortcut syntax for transactions with just one participant:

result = pmnc.transaction.some_resource.some_method(...)

instead of previously used

xa = pmnc.transaction.create()
xa.some_resource.some_method(...)
result = xa.execute()[0]

• Added facility for increasing logging level locally, useful for debugging:

with pmnc.log.level("DEBUG"):
    pmnc.log.debug("now you see me")
pmnc.log.debug("now you don't")

• Added support for MySQL databases.

• Added support for sending e-mails with alternative representations.

• Added timedelta/interval data type support to PostgreSQL.

• Added EXECUTE query support to Oracle.

• Added automatic response gzip'ping to HTTP interface.

• Fixes to SMPP protocol to handle problems such as disconnects better.

• Performance optimizations for large amounts of data for all TCP-based protocols.

• Messages sent to a health monitor are duplicated in local log file.

• Fixed timeout bug in retry queue machinery.

• Python 3.3 compatibility fixes.

• Logging priority can be changed at runtime, no need to restart a cage.

• Request timeouts can be specified on per-interface basis. This way for example requests from a synchronous HTTP interface may have their deadline in 10 seconds, and requests for processing incoming JMS messages - in 30 seconds.

• Resource pools can be configured to be kept warm, i.e. having a few free instances always connected.

• Changed the way logging works. Now each request has a readable description which is included in its logging messages. Request's remaining time to live is also included.

• Added support for custom acceptance methods for transactions. Acceptance method receives intermediate results of the resources participating in the transaction and decides what to do with them. It can perform tricks such as:
1. Pick the fastest result and return transaction result early without waiting for all participants to return.
2. Deduce the result by voting despite occasional failures.
3. Mask irrelevant failures.
etc.

• Changed the way transactions and resources report errors. Now if a transaction fails, it throws ResourceError, which contains failure information in a uniform way.

• Retried call queues can be configured to maintain FIFO order, a failed call will remain in front of the queue and retried.

• Added configurable file permissions when files are being saved to a shared directory using file resource.

• Email protocol now supports sending (SMTP) and receiving (POP3) messages over SSL.

• Oracle SQL resources return all numbers as Decimals, never ints, making no attempt to deduce type information from the values.

• Unknown SMPP optional tags on incoming PDUs are ignored, rather than cause a parsing failure. One awkward SMPP SMSC kept sending optional tags from SMPP 5.0 to SMPP 3.4 client which Pythomnic3k is.

• GREATLY simplified the request creation logic. This is of use for anyone who is going to implement some protocol support for Pythomnic3k.

• Added CPU, memory statistics to cage performance web page.

• Request context is preserved between cages verbatim, not modified across RPC calls.

• Persistent state is disabled if free disk space falls below configured threshold. Note that this affects retried calls too.

• Python 3.2 compatibility fixes.

• Added support for "reverse RPC" where target cage polls the source for incoming calls. This is useful in DMZ scenarios when a service from DMZ needs to call the one from LAN, but TCP connections from DMZ to LAN are prohibited by the firewall.

• Switched persistent RPC message sending syntax from

pmnc("target_cage:retry").module.method()

to

pmnc("target_cage", queue = "retry").module.method()

• Support multiple persistent retry queues using

pmnc("target_cage", queue = "queue_name")

where each queue may have its own retry interval and expiration settings.

• Results of SQL requests (recordsets) returned from database resources are no longer lists of field name-value dicts, but instead lists of instances of special class SQLRecord which contains the same name-value pairs, but allows case-insensitive field reference so that

rec["name"] == rec["NAME"] == rec["nAmE"]

this is useful with Oracle.

• HTTP protocol request handler now has control over cache-related response header fields.

• Performance monitoring web page now displays configurable amount of application messages, not always last 100 as before.