U a@sdZddlZddlmZddlmZddlmZddlmZddlmZdd l m Z dd l m Z dd l mZGd d d eZGdddeZGdddeZddZeGdddeeZGdddeeZGdddeeZdS)a*2Provide support for tracking of in-place changes to scalar values, which are propagated into ORM change events on owning parent objects. .. _mutable_scalars: Establishing Mutability on Scalar Column Values =============================================== A typical example of a "mutable" structure is a Python dictionary. Following the example introduced in :ref:`types_toplevel`, we begin with a custom type that marshals Python dictionaries into JSON strings before being persisted:: from sqlalchemy.types import TypeDecorator, VARCHAR import json class JSONEncodedDict(TypeDecorator): "Represents an immutable structure as a json-encoded string." impl = VARCHAR def process_bind_param(self, value, dialect): if value is not None: value = json.dumps(value) return value def process_result_value(self, value, dialect): if value is not None: value = json.loads(value) return value The usage of ``json`` is only for the purposes of example. The :mod:`sqlalchemy.ext.mutable` extension can be used with any type whose target Python type may be mutable, including :class:`.PickleType`, :class:`_postgresql.ARRAY`, etc. When using the :mod:`sqlalchemy.ext.mutable` extension, the value itself tracks all parents which reference it. Below, we illustrate a simple version of the :class:`.MutableDict` dictionary object, which applies the :class:`.Mutable` mixin to a plain Python dictionary:: from sqlalchemy.ext.mutable import Mutable class MutableDict(Mutable, dict): @classmethod def coerce(cls, key, value): "Convert plain dictionaries to MutableDict." if not isinstance(value, MutableDict): if isinstance(value, dict): return MutableDict(value) # this call will raise ValueError return Mutable.coerce(key, value) else: return value def __setitem__(self, key, value): "Detect dictionary set events and emit change events." dict.__setitem__(self, key, value) self.changed() def __delitem__(self, key): "Detect dictionary del events and emit change events." dict.__delitem__(self, key) self.changed() The above dictionary class takes the approach of subclassing the Python built-in ``dict`` to produce a dict subclass which routes all mutation events through ``__setitem__``. There are variants on this approach, such as subclassing ``UserDict.UserDict`` or ``collections.MutableMapping``; the part that's important to this example is that the :meth:`.Mutable.changed` method is called whenever an in-place change to the datastructure takes place. We also redefine the :meth:`.Mutable.coerce` method which will be used to convert any values that are not instances of ``MutableDict``, such as the plain dictionaries returned by the ``json`` module, into the appropriate type. Defining this method is optional; we could just as well created our ``JSONEncodedDict`` such that it always returns an instance of ``MutableDict``, and additionally ensured that all calling code uses ``MutableDict`` explicitly. When :meth:`.Mutable.coerce` is not overridden, any values applied to a parent object which are not instances of the mutable type will raise a ``ValueError``. Our new ``MutableDict`` type offers a class method :meth:`~.Mutable.as_mutable` which we can use within column metadata to associate with types. This method grabs the given type object or class and associates a listener that will detect all future mappings of this type, applying event listening instrumentation to the mapped attribute. Such as, with classical table metadata:: from sqlalchemy import Table, Column, Integer my_data = Table('my_data', metadata, Column('id', Integer, primary_key=True), Column('data', MutableDict.as_mutable(JSONEncodedDict)) ) Above, :meth:`~.Mutable.as_mutable` returns an instance of ``JSONEncodedDict`` (if the type object was not an instance already), which will intercept any attributes which are mapped against this type. Below we establish a simple mapping against the ``my_data`` table:: from sqlalchemy import mapper class MyDataClass(object): pass # associates mutation listeners with MyDataClass.data mapper(MyDataClass, my_data) The ``MyDataClass.data`` member will now be notified of in place changes to its value. There's no difference in usage when using declarative:: from sqlalchemy.ext.declarative import declarative_base Base = declarative_base() class MyDataClass(Base): __tablename__ = 'my_data' id = Column(Integer, primary_key=True) data = Column(MutableDict.as_mutable(JSONEncodedDict)) Any in-place changes to the ``MyDataClass.data`` member will flag the attribute as "dirty" on the parent object:: >>> from sqlalchemy.orm import Session >>> sess = Session() >>> m1 = MyDataClass(data={'value1':'foo'}) >>> sess.add(m1) >>> sess.commit() >>> m1.data['value1'] = 'bar' >>> assert m1 in sess.dirty True The ``MutableDict`` can be associated with all future instances of ``JSONEncodedDict`` in one step, using :meth:`~.Mutable.associate_with`. This is similar to :meth:`~.Mutable.as_mutable` except it will intercept all occurrences of ``MutableDict`` in all mappings unconditionally, without the need to declare it individually:: MutableDict.associate_with(JSONEncodedDict) class MyDataClass(Base): __tablename__ = 'my_data' id = Column(Integer, primary_key=True) data = Column(JSONEncodedDict) Supporting Pickling -------------------- The key to the :mod:`sqlalchemy.ext.mutable` extension relies upon the placement of a ``weakref.WeakKeyDictionary`` upon the value object, which stores a mapping of parent mapped objects keyed to the attribute name under which they are associated with this value. ``WeakKeyDictionary`` objects are not picklable, due to the fact that they contain weakrefs and function callbacks. In our case, this is a good thing, since if this dictionary were picklable, it could lead to an excessively large pickle size for our value objects that are pickled by themselves outside of the context of the parent. The developer responsibility here is only to provide a ``__getstate__`` method that excludes the :meth:`~MutableBase._parents` collection from the pickle stream:: class MyMutableType(Mutable): def __getstate__(self): d = self.__dict__.copy() d.pop('_parents', None) return d With our dictionary example, we need to return the contents of the dict itself (and also restore them on __setstate__):: class MutableDict(Mutable, dict): # .... def __getstate__(self): return dict(self) def __setstate__(self, state): self.update(state) In the case that our mutable value object is pickled as it is attached to one or more parent objects that are also part of the pickle, the :class:`.Mutable` mixin will re-establish the :attr:`.Mutable._parents` collection on each value object as the owning parents themselves are unpickled. Receiving Events ---------------- The :meth:`.AttributeEvents.modified` event handler may be used to receive an event when a mutable scalar emits a change event. This event handler is called when the :func:`.attributes.flag_modified` function is called from within the mutable extension:: from sqlalchemy.ext.declarative import declarative_base from sqlalchemy import event Base = declarative_base() class MyDataClass(Base): __tablename__ = 'my_data' id = Column(Integer, primary_key=True) data = Column(MutableDict.as_mutable(JSONEncodedDict)) @event.listens_for(MyDataClass.data, "modified") def modified_json(instance): print("json value modified:", instance.data) .. _mutable_composites: Establishing Mutability on Composites ===================================== Composites are a special ORM feature which allow a single scalar attribute to be assigned an object value which represents information "composed" from one or more columns from the underlying mapped table. The usual example is that of a geometric "point", and is introduced in :ref:`mapper_composite`. As is the case with :class:`.Mutable`, the user-defined composite class subclasses :class:`.MutableComposite` as a mixin, and detects and delivers change events to its parents via the :meth:`.MutableComposite.changed` method. In the case of a composite class, the detection is usually via the usage of Python descriptors (i.e. ``@property``), or alternatively via the special Python method ``__setattr__()``. Below we expand upon the ``Point`` class introduced in :ref:`mapper_composite` to subclass :class:`.MutableComposite` and to also route attribute set events via ``__setattr__`` to the :meth:`.MutableComposite.changed` method:: from sqlalchemy.ext.mutable import MutableComposite class Point(MutableComposite): def __init__(self, x, y): self.x = x self.y = y def __setattr__(self, key, value): "Intercept set events" # set the attribute object.__setattr__(self, key, value) # alert all parents to the change self.changed() def __composite_values__(self): return self.x, self.y def __eq__(self, other): return isinstance(other, Point) and \ other.x == self.x and \ other.y == self.y def __ne__(self, other): return not self.__eq__(other) The :class:`.MutableComposite` class uses a Python metaclass to automatically establish listeners for any usage of :func:`_orm.composite` that specifies our ``Point`` type. Below, when ``Point`` is mapped to the ``Vertex`` class, listeners are established which will route change events from ``Point`` objects to each of the ``Vertex.start`` and ``Vertex.end`` attributes:: from sqlalchemy.orm import composite, mapper from sqlalchemy import Table, Column vertices = Table('vertices', metadata, Column('id', Integer, primary_key=True), Column('x1', Integer), Column('y1', Integer), Column('x2', Integer), Column('y2', Integer), ) class Vertex(object): pass mapper(Vertex, vertices, properties={ 'start': composite(Point, vertices.c.x1, vertices.c.y1), 'end': composite(Point, vertices.c.x2, vertices.c.y2) }) Any in-place changes to the ``Vertex.start`` or ``Vertex.end`` members will flag the attribute as "dirty" on the parent object:: >>> from sqlalchemy.orm import Session >>> sess = Session() >>> v1 = Vertex(start=Point(3, 4), end=Point(12, 15)) >>> sess.add(v1) >>> sess.commit() >>> v1.end.x = 8 >>> assert v1 in sess.dirty True Coercing Mutable Composites --------------------------- The :meth:`.MutableBase.coerce` method is also supported on composite types. In the case of :class:`.MutableComposite`, the :meth:`.MutableBase.coerce` method is only called for attribute set operations, not load operations. Overriding the :meth:`.MutableBase.coerce` method is essentially equivalent to using a :func:`.validates` validation routine for all attributes which make use of the custom composite type:: class Point(MutableComposite): # other Point methods # ... def coerce(cls, key, value): if isinstance(value, tuple): value = Point(*value) elif not isinstance(value, Point): raise ValueError("tuple or Point expected") return value Supporting Pickling -------------------- As is the case with :class:`.Mutable`, the :class:`.MutableComposite` helper class uses a ``weakref.WeakKeyDictionary`` available via the :meth:`MutableBase._parents` attribute which isn't picklable. If we need to pickle instances of ``Point`` or its owning class ``Vertex``, we at least need to define a ``__getstate__`` that doesn't include the ``_parents`` dictionary. Below we define both a ``__getstate__`` and a ``__setstate__`` that package up the minimal form of our ``Point`` class:: class Point(MutableComposite): # ... def __getstate__(self): return self.x, self.y def __setstate__(self, state): self.x, self.y = state As with :class:`.Mutable`, the :class:`.MutableComposite` augments the pickling process of the parent's object-relational state so that the :meth:`MutableBase._parents` collection is restored to all ``Point`` objects. N)event)inspect)types)Mapper)mapper) flag_modified)SchemaEventTarget)memoized_propertyc@s@eZdZdZeddZeddZeddZedd Z d S) MutableBasezPCommon base class to :class:`.Mutable` and :class:`.MutableComposite`. cCstS)aDictionary of parent object's :class:`.InstanceState`->attribute name on the parent. This attribute is a so-called "memoized" property. It initializes itself with a new ``weakref.WeakKeyDictionary`` the first time it is accessed, returning the same object upon subsequent access. .. versionchanged:: 1.4 the :class:`.InstanceState` is now used as the key in the weak dictionary rather than the instance itself. )weakrefWeakKeyDictionaryselfrWC:\Users\vtejo\AppData\Local\Temp\pip-unpacked-wheel-nyjtotrf\sqlalchemy\ext\mutable.py_parentswszMutableBase._parentscCs(|dkr dSd}t||t|fdS)aGiven a value, coerce it into the target type. Can be overridden by custom subclasses to coerce incoming data into a particular type. By default, raises ``ValueError``. This method is called in different scenarios depending on if the parent class is of type :class:`.Mutable` or of type :class:`.MutableComposite`. In the case of the former, it is called for both attribute-set operations as well as during ORM loading operations. For the latter, it is only called during attribute-set operations; the mechanics of the :func:`.composite` construct handle coercion during load operations. :param key: string name of the ORM-mapped attribute being set. :param value: the incoming value. :return: the method should return the coerced value, or raise ``ValueError`` if the coercion cannot be completed. Nz1Attribute '%s' does not accept objects of type %s) ValueErrortype)clskeyvaluemsgrrrcoerceszMutableBase.coercecCs|jhS)aGiven a descriptor attribute, return a ``set()`` of the attribute keys which indicate a change in the state of this attribute. This is normally just ``set([attribute.key])``, but can be overridden to provide for additional keys. E.g. a :class:`.MutableComposite` augments this set with the attribute keys associated with the columns that comprise the composite value. This collection is consulted in the case of intercepting the :meth:`.InstanceEvents.refresh` and :meth:`.InstanceEvents.refresh_flush` events, which pass along a list of attribute names that have been refreshed; the list is compared against this set to determine if action needs to be taken. .. versionadded:: 1.0.5 rr attributerrr_get_listen_keysszMutableBase._get_listen_keyscs|j||jk rdS|j}|fddfdd}fdd}fdd }fd d }tj|d d d dtj|d|d d dtj|d|d d dtj|d|d d d dtj|d|d d dtj|d|d d ddS)]Establish this type as a mutation listener for the given mapped descriptor. Ncs>|jd}|dk r:r0|}||j<|j|<dS)zListen for objects loaded or refreshed. Wrap the target data member's value with ``Mutable``. N)dictgetrr)stateargsval)rrrrrloads   z.MutableBase._listen_on_attribute..loadcs|r|r|dSN) intersection)r!ctxattrs) listen_keysr$rr load_attrssz4MutableBase._listen_on_attribute..load_attrscsT||kr |St|s"|}|dk r4|j|<t|rP|jt|d|S)zListen for set/replace events on the target data member. Establish a weak reference to the parent object on the incoming value, remove it for the one outgoing. N) isinstancerrpopr)targetrZoldvalueZ initiator)rrrrset_s     z.MutableBase._listen_on_attribute..set_cs8|jd}|dk r4d|kr&g|d<|d|dSNzext.mutable.values)rr appendr!Z state_dictr#rrrpickles z0MutableBase._listen_on_attribute..picklecs$d|kr |dD]}|j|<qdSr/)rr1rrrunpickles z2MutableBase._listen_on_attribute..unpickler$T)raw propagateZrefreshZ refresh_flushset)r4retvalr5r2r3)rclass_rrlisten)rrrZ parent_clsr*r.r2r3r)rrrr)r$r_listen_on_attributesR    z MutableBase._listen_on_attributeN) __name__ __module__ __qualname____doc__r r classmethodrrr:rrrrr qs   r c@s<eZdZdZddZeddZeddZedd Zd S) MutablezMixin that defines transparent propagation of change events to a parent object. See the example in :ref:`mutable_scalars` for usage information. cCs&|jD]\}}t||q dSz@Subclasses should call this method whenever change events occur.N)ritemsrobj)rparentrrrrchangedszMutable.changedcCs||d|jdS)rTN)r:r8rrrrassociate_with_attributesz Mutable.associate_with_attributecs fdd}ttd|dS)aAssociate this wrapper with all future mapped columns of the given type. This is a convenience method that calls ``associate_with_attribute`` automatically. .. warning:: The listeners established by this method are *global* to all mappers, and are *not* garbage collected. Only use :meth:`.associate_with` for types that are permanent to an application, not with ad-hoc types else this will cause unbounded growth in memory usage. cs>|jr dS|jD](}t|jdjrt||jqdS)Nr) non_primary column_attrsr+columnsrrFgetattrrrr8proprsqltyperrlisten_for_type3s  z/Mutable.associate_with..listen_for_typemapper_configuredN)rr9r)rrNrOrrMrassociate_with!szMutable.associate_withcsTtttr.tddd}dndfdd}ttd|S) aAssociate a SQL type with this mutable Python type. This establishes listeners that will detect ORM mappings against the given type, adding mutation event trackers to those mappings. The type is returned, unconditionally as an instance, so that :meth:`.as_mutable` can be used inline:: Table('mytable', metadata, Column('id', Integer, primary_key=True), Column('data', MyMutableType.as_mutable(PickleType)) ) Note that the returned type is always an instance, even if a class is given, and that only columns which are declared specifically with that type instance receive additional instrumentation. To associate a particular mutable type with all occurrences of a particular type, use the :meth:`.Mutable.associate_with` classmethod of the particular :class:`.Mutable` subclass to establish a global association. .. warning:: The listeners established by this method are *global* to all mappers, and are *not* garbage collected. Only use :meth:`.as_mutable` for types that are permanent to an application, not with ad-hoc types else this will cause unbounded growth in memory usage. Zbefore_parent_attachcSs||jd<dS)N_ext_mutable_orig_type)info)ZsqltyprDrrr_add_column_memodsz,Mutable.as_mutable.._add_column_memoTFcs^|jr dS|jD]H}r6t|jdr6|jjdksF|jdjkrt ||j qdS)NrSrRr) rGrHhasattrZ expressionrSr rIrrFrJrrKrZschema_event_checkrNrrrOls   z+Mutable.as_mutable..listen_for_typerP)rZ to_instancer+r rZ listens_forr9r)rrNrTrOrrVr as_mutable<s!     zMutable.as_mutableN) r;r<r=r>rEr?rFrQrWrrrrr@ s  r@c@s$eZdZdZeddZddZdS)MutableCompositezMixin that defines transparent propagation of change events on a SQLAlchemy "composite" object to its owning parent or parents. See the example in :ref:`mutable_composites` for usage information. cCs|jh|jjSr%)runionproperty_attribute_keysrrrrrsz!MutableComposite._get_listen_keyscCsN|jD]>\}}|j|}t||jD]\}}t|||q.q dSrA) rrBrZ get_propertyzipZ__composite_values__r[setattrrC)rrDrrLr attr_namerrrrEs  zMutableComposite.changedN)r;r<r=r>r?rrErrrrrX}s rXcCs(dd}ttd|s$ttd|dS)NcSsJ|jD]>}t|drt|jtrt|jtr|jt||j d|qdS)Ncomposite_classF) Ziterate_propertiesrUr+r_r issubclassrXr:rJrrKrrr_listen_for_types    z3_setup_composite_listener.._listen_for_typerP)rcontainsrr9)rarrr_setup_composite_listeners rcc@sdeZdZdZddZddZddZdd Zd d Zd d Z ddZ e ddZ ddZ ddZdS) MutableDictajA dictionary type that implements :class:`.Mutable`. The :class:`.MutableDict` object implements a dictionary that will emit change events to the underlying mapping when the contents of the dictionary are altered, including when values are added or removed. Note that :class:`.MutableDict` does **not** apply mutable tracking to the *values themselves* inside the dictionary. Therefore it is not a sufficient solution for the use case of tracking deep changes to a *recursive* dictionary structure, such as a JSON structure. To support this use case, build a subclass of :class:`.MutableDict` that provides appropriate coercion to the values placed in the dictionary so that they too are "mutable", and emit events up to their parent structure. .. seealso:: :class:`.MutableList` :class:`.MutableSet` cCst||||dS)z4Detect dictionary set events and emit change events.N)r __setitem__rE)rrrrrrreszMutableDict.__setitem__cCst|||}||Sr%)r setdefaultrE)rrrresultrrrrfszMutableDict.setdefaultcCst|||dS)z4Detect dictionary del events and emit change events.N)r __delitem__rE)rrrrrrhs zMutableDict.__delitem__cOstj|f|||dSr%)rupdaterE)rakwrrrriszMutableDict.updatecGstj|f|}||Sr%)rr,rErargrgrrrr,szMutableDict.popcCst|}||Sr%)rpopitemrE)rrgrrrrns zMutableDict.popitemcCst||dSr%)rclearrErrrrros zMutableDict.clearcCs0t||s(t|tr||St||S|SdS)z3Convert plain dictionary to instance of this class.N)r+rr@r)rrrrrrrs    zMutableDict.coercecCst|Sr%)rrrrr __getstate__szMutableDict.__getstate__cCs||dSr%rirr!rrr __setstate__szMutableDict.__setstate__N)r;r<r=r>rerfrhrir,rnror?rrprsrrrrrds rdc@seZdZdZddZddZddZdd Zd d Zd d Z ddZ ddZ ddZ ddZ ddZddZddZddZddZed d!Zd"S)# MutableListajA list type that implements :class:`.Mutable`. The :class:`.MutableList` object implements a list that will emit change events to the underlying mapping when the contents of the list are altered, including when values are added or removed. Note that :class:`.MutableList` does **not** apply mutable tracking to the *values themselves* inside the list. Therefore it is not a sufficient solution for the use case of tracking deep changes to a *recursive* mutable structure, such as a JSON structure. To support this use case, build a subclass of :class:`.MutableList` that provides appropriate coercion to the values placed in the dictionary so that they too are "mutable", and emit events up to their parent structure. .. versionadded:: 1.1 .. seealso:: :class:`.MutableDict` :class:`.MutableSet` cCs|jt|ffSr% __class__listrprotorrr __reduce_ex__ szMutableList.__reduce_ex__cCs||dd<dSr%rrrrrrrsszMutableList.__setstate__cCst||||dSz.Detect list set events and emit change events.N)rwrerE)rindexrrrrreszMutableList.__setitem__cCst|||||dSr{)rw __setslice__rE)rstartendrrrrr}szMutableList.__setslice__cCst|||dSz.Detect list del events and emit change events.N)rwrhrE)rr|rrrrhs zMutableList.__delitem__cCst||||dSr)rw __delslice__rE)rr~rrrrr"szMutableList.__delslice__cGstj|f|}||Sr%)rwr,rErlrrrr,'szMutableList.popcCst|||dSr%)rwr0rErxrrrr0,s zMutableList.appendcCst|||dSr%)rwextendrErrrrr0s zMutableList.extendcCs|||Sr%)rrrrr__iadd__4s zMutableList.__iadd__cCst||||dSr%)rwinsertrE)rirrrrr8szMutableList.insertcCst|||dSr%)rwremoverE)rrrrrr<s zMutableList.removecCst||dSr%)rwrorErrrrro@s zMutableList.clearcKstj|f||dSr%)rwsortrE)rrkrrrrDszMutableList.sortcCst||dSr%)rwreverserErrrrrHs zMutableList.reversecCs0t||s(t|tr||St||S|SdS)z-Convert plain list to instance of this class.N)r+rwr@rrr|rrrrrLs    zMutableList.coerceN)r;r<r=r>rzrsrer}rhrr,r0rrrrrorrr?rrrrrrts$rtc@seZdZdZddZddZddZdd Zd d Zd d Z ddZ ddZ ddZ ddZ ddZddZddZeddZddZd d!Zd"d#Zd$S)% MutableSetaKA set type that implements :class:`.Mutable`. The :class:`.MutableSet` object implements a set that will emit change events to the underlying mapping when the contents of the set are altered, including when values are added or removed. Note that :class:`.MutableSet` does **not** apply mutable tracking to the *values themselves* inside the set. Therefore it is not a sufficient solution for the use case of tracking deep changes to a *recursive* mutable structure. To support this use case, build a subclass of :class:`.MutableSet` that provides appropriate coercion to the values placed in the dictionary so that they too are "mutable", and emit events up to their parent structure. .. versionadded:: 1.1 .. seealso:: :class:`.MutableDict` :class:`.MutableList` cGstj|f||dSr%)r6rirErrmrrrriqszMutableSet.updatecGstj|f||dSr%)r6intersection_updaterErrrrruszMutableSet.intersection_updatecGstj|f||dSr%)r6difference_updaterErrrrryszMutableSet.difference_updatecGstj|f||dSr%)r6symmetric_difference_updaterErrrrr}sz&MutableSet.symmetric_difference_updatecCs|||Sr%rqrotherrrr__ior__s zMutableSet.__ior__cCs|||Sr%)rrrrr__iand__s zMutableSet.__iand__cCs|||Sr%)rrrrr__ixor__s zMutableSet.__ixor__cCs|||Sr%)rrrrr__isub__s zMutableSet.__isub__cCst|||dSr%)r6addrErelemrrrrs zMutableSet.addcCst|||dSr%)r6rrErrrrrs zMutableSet.removecCst|||dSr%)r6discardrErrrrrs zMutableSet.discardcGstj|f|}||Sr%)r6r,rErlrrrr,szMutableSet.popcCst||dSr%)r6rorErrrrros zMutableSet.clearcCs0t||s(t|tr||St||S|SdS)z,Convert plain set to instance of this class.N)r+r6r@rrrrrrs    zMutableSet.coercecCst|Sr%)r6rrrrrpszMutableSet.__getstate__cCs||dSr%rqrrrrrrsszMutableSet.__setstate__cCs|jt|ffSr%rurxrrrrzszMutableSet.__reduce_ex__N)r;r<r=r>rirrrrrrrrrrr,ror?rrprsrzrrrrrWs& r)r>r rrrZormrrZorm.attributesrZsql.baser utilr objectr r@rXrcrrdrwrtr6rrrrrs(_        rIe