.. @+leo-ver=5-thin
.. @+node:ekr.20100119205347.6015: * @file ../doc/leoToDo.txt
.. @@language rest # override the default .txt coloring.

.. @+all
.. @+node:ekr.20101129095441.5963: ** Improve screenshots page
.. @+node:ekr.20110609161752.16461: ** Post to pyxides re "your mission"
.. @+node:ekr.20110616084347.14800: *3* Post to pyxides, 2011/06/18
http://groups.google.com/group/pyxides

On 2010/02/23 I commented about Leo 4.7 as it relates to Robin Dunn's post,
http://groups.google.com/group/leo-editor/browse_thread/thread/4f76a0f57759aba

Imo, Leo has accomplished the mission. Leo has all the important features that
Leo's users have requested. Yes, wishlist items remain. See:
https://bugs.launchpad.net/leo-editor/+bugs 

None of these wish-list items interferes in any way with Leo's day-to-day
operation. Furthermore, many of Leo's essential features moot the need for more
traditional features.

For example, Leo 4.9 adds full support for macros. Recording, saving, editing
and retrieving macros is easier in Leo than in other editors because Leo stores
macros in @macro nodes, not external files. But few, if any, of Leo's users are
likely to use macros because Leo's @button nodes make all of Python's scripting
abilities easily available on a node-by-node or outline-wide basis.

It may be that Leo could benefit from some Emacs-like or vim-like features, but
that doesn't seem so likely.  Leo has many users who also use Emacs and vim, and
they seem happy enough :-)

Finally, Leo offers features that have no counterpart at all in editors like
Emacs and vim. For example, the rst3 command converts an outline to restructured
text. See: http://webpages.charter.net/edreamleo/rstplugin3.html Yes, one could
imagine an org-mode command that does this, but the fact is that Leo's outline
orientation has given it abilities possessed by no other editor or IDE.

Naturally, you may disagree. I invite you to try Leo. If, after using Leo for
real work, you find you would like some new feature, then by all means ask.

Edward
.. @+node:ekr.20110609102702.16471: ** To do: 4.10
.. @+node:ekr.20110620065017.14900: *3* Bugs
.. @+node:ekr.20110620122650.14904: *4* Make it clearer in the Edit:Find menu how to do search/replace
.. @+node:ekr.20110620083705.14901: *4* Fix or disable curses and none gui's
See scanOptions.
.. @+node:ekr.20110619173515.14895: *4* Fix bug: realpath in g.openWithFileName
@nocolor-node

http://groups.google.com/group/leo-editor/browse_thread/thread/cd5e2125c9e13d8f

g.openWithFileName needs

   fileName = g.os_path_realpath(fileName)

at the start of it, otherwise

g.openWithFileName("/home/tbrown/.leo/.todo.leo", c) will open another
instance of an already open "/mnt/usr1/usr1/home/tbrown/.leo/.todo.leo"
because it doesn't recognize them as being the same.

It would be nice if the original attempt to open
"/home/tbrown/.leo/.todo.leo" was not converted to
"/mnt/usr1/usr1/home/tbrown/.leo/.todo.leo", but I think the POSIX
spec. insists on giving CPython the real path form, so this is hard to
avoid.  You see this when selecting a file using a file dialog, in the
dialog you choose /home/me/myfile.txt, but the return
is /mnt/usr1/usr1/home/me/myfile.txt.

Anyway, let me know if and when I can push that to trunk, impacts seem
to be limited to (a) not failing to detect an already opened file in
g.openWithFileName, which is a bug, and (b) sometimes less attractive
apparent paths, which is annoying, but I think better than the bug.

(this of course trips up UNLs
if /mnt/usr1/usr1/home/tbrown/.leo/.todo.leo is already loaded and the
UNL is /home/tbrown/.leo/.todo.leo#Home-->Paint roof)
.. @+node:ekr.20110619173515.14896: *4* Fix bug: wrong modality level on autocompleter
Just noticed the autocompleter pop-up is modal globally, not just for
the Leo windows.  Probably should only block the Leo windows.
.. @+node:ekr.20110620065017.14898: *4* Fix Coloring problem with move-lines-up
.. @+node:ekr.20051214133130.1: *5* endCommand
# New in Leo 4.4b4: calling endCommand is valid for all widgets,
# but handles undo only if we are in body pane.

def endCommand(self,label=None,changed=True,setLabel=True):

    '''Do the common processing at the end of each command.'''

    trace =  False and not g.unitTesting
    c = self.c ; b = self.undoData ; k = self.k

    if trace: g.trace('changed',changed)

    if b and b.name.startswith('body') and changed:
        c.frame.body.onBodyChanged(undoType=b.undoType,
            oldSel=b.oldSel,oldText=b.oldText,oldYview=None)

    self.undoData = None # Bug fix: 1/6/06 (after a5 released).

    k.clearState()

    # Warning: basic editing commands **must not** set the label.
    if setLabel:
        if label:
            k.setLabelGrey(label)
        else:
            k.resetLabel()
.. @+node:ekr.20031218072017.1329: *5* onBodyChanged (leoBody)
# This is the only key handler for the body pane.
def onBodyChanged (self,undoType,oldSel=None,oldText=None,oldYview=None):

    '''Update Leo after the body has been changed.'''

    trace = False and not g.unitTesting
    body = self ; c = self.c
    bodyCtrl = w = body.bodyCtrl
    p = c.p
    insert = w.getInsertPoint()
    ch = g.choose(insert==0,'',w.get(insert-1))
    ch = g.toUnicode(ch)
    newText = w.getAllText() # Note: getAllText converts to unicode.
    newSel = w.getSelectionRange()
    if not oldText:
        oldText = p.b ; changed = True
    else:
        changed = oldText != newText
    if not changed: return
    if trace:
        # g.trace(repr(ch),'changed:',changed,'newText:',len(newText),'w',w)
        g.trace('oldSel',oldSel,'newSel',newSel)
    c.undoer.setUndoTypingParams(p,undoType,
        oldText=oldText,newText=newText,oldSel=oldSel,newSel=newSel,oldYview=oldYview)
    p.v.setBodyString(newText)
    p.v.insertSpot = body.getInsertPoint()
    << recolor the body >>
    if not c.changed: c.setChanged(True)
    self.updateEditors()
    << update icons if necessary >>
.. @+node:ekr.20051026083733.6: *6* << recolor the body >>
body.colorizer.interrupt()
c.frame.scanForTabWidth(p)
body.recolor(p,incremental=not self.forceFullRecolorFlag)
self.forceFullRecolorFlag = False

if g.app.unitTesting:
    g.app.unitTestDict['colorized'] = True
.. @+node:ekr.20051026083733.7: *6* << update icons if necessary >>

redraw_flag = False
# Update dirty bits.
# p.setDirty() sets all cloned and @file dirty bits.
if not p.isDirty() and p.setDirty():
    redraw_flag = True

# Update icons. p.v.iconVal may not exist during unit tests.
val = p.computeIcon()
# g.trace('new val:',val,'old val:',hasattr(p.v,'iconVal') and p.v.iconVal or '<None>')
if not hasattr(p.v,"iconVal") or val != p.v.iconVal:
    p.v.iconVal = val
    redraw_flag = True

if redraw_flag:
    c.redraw_after_icons_changed()
.. @+node:ekr.20110620065017.14899: *5* move-lines-up bug
@language python

        @language python


        def test():

            """ a
            b
            c
            print 1
            """

            print 2
.. @+node:ekr.20110617144701.14907: *3* Code cleaning
@nocolor-node

- Remove multiple continguous comment lines.

- Eliminate k.callStateFunction if possible.
  The code should just call the "func":
  it should be up to each mode handler to handle special cases!
.. @+node:ekr.20110614123640.6588: *3* Documentation re emacs-like commands for editing body text
@nocolor-node

To me a higher priority would be documentation of, and if necessary
improvement of, the commands for editing the body text, moving the
position, selection, etc.

.. @+node:ekr.20110614123640.6586: *3* script to check myLeoSettings.leo
@nocolor-node


http://groups.google.com/group/leo-editor/browse_thread/thread/764c9d2f260e2441/eb0564983b90feb5

Perhaps leo could also have a command to validate myLeoSettings.leo
and complain if it finds some obsolete settings that got removed from
leo, but are still present in myLeoSettings. A special case would be
to validate enabled-plugins node too, which can sometimes contain
legacy lines, like it recently happened with UNL plugin that got moved
to contrib branch.
.. @+node:ekr.20110611043506.16494: *3* Ashland sprint items: 4
.. @+node:ekr.20110527084258.18378: *4* New file format
@nocolor-node

** remove window state, expansion status etc.
   stuff from .leo file, and move that over to c.db

- <attr> solves pickle problem.

* Treat gnx's as strings: don't parse them.
  http://mail.google.com/mail/#inbox/12f3d4950fbabeea
  
* Don't save expansion bits in uA if not saving expansion bits. It's illogical
  to save bits in uA's if they aren't save in in the <v> elements.
  
    @bool put_expansion_bits_in_leo_files = False

- Use uuid's?

- Remove spaces from user names.

.. @+node:ekr.20110609042343.16546: *5* Notes
.. @+node:ekr.20110609042343.16548: *6* File format, v3 draft 5
@language rest
@pagewidth 65

http://groups.google.com/group/leo-editor/browse_thread/thread/2ddb57c62e67825c

Leo's file format: version 3, draft 5

This draft is intended to reflect minimal changes from Leo's
present file format, but with improvements mentioned at the
Ashland sprint and developed afterward.

This draft covers only Leo's xml format, but it may be adapted
for use as a json file format.

This draft contains significant invention by EKR. See the next
section. Don't panic: it can be changed :-)

Summary of changes from draft 4
===============================

- Elements will be renamed as follows::

    old         new
    ====        ====
    <vnodes>    <directed-acyclic-graph>
    <tnodes>    <data-list>
    <v>         <node>
    <t>         <data>

- Nesting of <node> elements represents the structure of the DAG,
  just as nesting of <v> elements does at present.
  
- Thus, there will be no <edge> elements.

- Headlines will move from <v> elements to <data> elements.
  This "normalizes" the data: headlines will appear only once.
  
- <attr> elements will represent uA's.  A full discussion below.

  Ideally, I would like to support only string and json formats
  for <attr> elements.  This is open to discussion. 

- Only <node> elements will contain <attr> elements.

- <node> elements for @file nodes will contain
  <at-file-attributes> elements, representing Leo's "hidden
  machinery" attributes.  <at-file-attributes> will contain
  <attr> elements. 

First lines
===========

Leo file will start with the following::

    <?xml version="1.0" encoding="utf-8"?>
    <?xml-stylesheet my_stylesheet?>
    <!-- Created by Leo (http://webpages.charter.net/edreamleo/front.html) -->
    <leo_file file_format="3"
        xmlns:leo="http://www.leo-editor.org/2011/leo"/>
        

No session data
===============

There will be no <globals>, <preferences> or
<find_panel_settings> elements. All such "session data" will be
restored from the cache, or from defaults if caching is disabled.

**Important**: there is no representation of expansion state or
the currently selected node anywhere in the .leo file.
Eliminating these data is contingent on having Leo work well with
caching disabled.

Note: marks are more than session data. They must appear within
<node> elements.


Summary of format
=================

.leo files will have the following form, with similar indentation::

    <?xml version="1.0" encoding="utf-8"?>
    <?xml-stylesheet my_stylesheet?>
    <!-- Created by Leo (http://webpages.charter.net/edreamleo/front.html) -->
    <leo_file file_format="3" xmlns:leo="http://www.leo-editor.org/2011/leo"/>
    <directed-acyclic-graph>
        <node id="gnx">
            <!-- contained node elements, if any.
        </node>
        <node id="gnx">
            <!-- contained v elements, if any.
        </node>
        ...
    </directed-acyclic-graph>
    <data-list>
        <data id="gnx">
            <!-- marked attribute appears only if the tnode/vnode is marked -->
            <head>headline text</head>
            <attr key="marked">1</attr>
            <!-- uA's... -->
            <!-- format="string" is the default -->
            <attr key="a">a string</attr>
            <attr key="b" format="json">a json string</attr>
            <attr key="c" format="pickle">a pickled string</attr>
            <attr key="d" format="binhex">a binhexed string</attr>
            ...
            <body>body text</body>
        </data>
        ...
    </data-list>
    </leo_file>

<attr> elements
===============

<attr> elements will one of the following forms::

    <attr key="a">a unicode string</attr>
    <attr key="b" format="json">a json string</attr>
    <attr key="c" format="pickle">a json string</attr>
    <attr key="d" format="binhex">a binhexed string</attr>
    
The value will be a string by default.

If possible, I would like to support only the string and json
formats. This would make the data as transparent as possible.
Please mentally amend the following discussion...

uA's that start with "binhex_" will use the binhex format. This
prefix must be retained in the type field, so the read code can
restore them.

If the value is not a string, and there is no "binhex_" prefix,
the write code will use format="json" if json.dumps succeeds, and
will use format="pickle" otherwise.

No <attr> element will be written if both json.dumps and
pickle.dumps fail. Attribute failures will create a warning for
the plugin developer.
    
Descendant attributes in @file trees
====================================

Descendants of @file nodes do not appear in .leo files. Thus,
important data must be stored in the so-called hidden machinery:
attributes of the @file node.

The <at-file-attributes> element may optionally be contained in
the <node> element for @file nodes::

    <at-file-attributes>
        <attr>key="ua-name"
            format="(empty)/json/pickle/binhex"
            gnx="gnx">value
        </attr>
        ...
    </at-file-attributes>
.. @+node:ekr.20110421132230.6107: *6* File format, v3 draft 4
@language rest
@pagewidth 65

Leo's file format: version 3, draft 4

http://groups.google.com/group/leo-editor/browse_thread/thread/a2b7e5321d62b64/a4cc51d404af94aa

Here is the latest version, with the graphml stuff removed.

This draft is intended to reflect our recent discussions, with no
new invention from me. All comments and corrections welcome.

This draft covers only Leo's xml format, but it may be adapted
for use as a json file format.

Recent changes
==============

- Removed graphml stuff, including leo: prefixes.

- Used "key" in <attr> elements (and "type" in <edge> elements.)

First lines
===========

Leo file will start with the following::

    <?xml version="1.0" encoding="utf-8"?>
    <?xml-stylesheet my_stylesheet?>
    <!-- Created by Leo (http://webpages.charter.net/edreamleo/front.html) -->
    <leo_file file_format="3"
        xmlns:leo="http://www.leo-editor.org/2011/leo"/>
        

No session data
===============

There will be no <globals>, <preferences> or
<find_panel_settings> elements. All such "session data" will be
restored from the cache, or from defaults if caching is disabled.

**Important**: there is no representation of expansion state or
the currently selected node anywhere in the .leo file.
Eliminating these data is contingent on having Leo work well with
caching disabled.

Note: marks are more than session data. They must appear
somewhere within <node> elements.


Summary of format
=================

.leo files will have the following form, with similar indentation::

    <?xml version="1.0" encoding="utf-8"?>
    <?xml-stylesheet my_stylesheet?>
    <!-- Created by Leo (http://webpages.charter.net/edreamleo/front.html) -->
    <leo_file file_format="3" xmlns:leo="http://www.leo-editor.org/2011/leo"/>
    <graph>
    <nodes>
        <!-- marked attribute appears only if the vnode is marked -->
        <node id="gnx"> 
            <head>headline text</head>
            <attr key="marked">1</attr>
            <!-- uA's... -->
            <!-- format="string" is the default -->
            <attr key="a">a string</attr>
            <attr key="b" format="json">a json string</attr>
            <attr key="c" format="pickle">a pickled string</attr>
            <attr key="d" format="binhex">a binhexed string</attr>
            ...
            <body>body text</body>
        </node>
        ...
        <!-- @file nodes contain a <data> package -->
        <node id="gnx">
            <head>@file x.py</head>
            ...
            <at-file-attributes>
                <attr>key="ua-name"
                    format="(empty)/json/pickle/binhex"
                    gnx="gnx">value
                </attr>
                ...
            </at-file-attributes>
        </node>
        ...
    </nodes>
    <edges>
        <edge type="child" from="gnx" to="gnx"</edge>
        ...
    </edges>
    </graph>
    </leo_file>

<attr> elements
===============

<attr> elements will one of the following forms::

    <attr key="a">a unicode string</attr>
    <attr key="b" format="json">a json string</attr>
    <attr key="c" format="pickle">a json string</attr>
    <attr key="d" format="binhex">a binhexed string</attr>
    
That is, the value will be a string by default.

uA's that start with "binhex_" will use the binhex format. This
prefix must be retained in the type field, so the read code can
restore them.

If the value is not a string, and there is no "binhex_" prefix,
the write code will use format="json" if json.dumps succeeds, and
will use format="pickle" otherwise.

No <attr> element will be written if both json.dumps and
pickle.dumps fail. Attribute failures will create a warning for
the plugin developer.

<edge> elements
===============

<edge> elements will have the form::

    <edge type="child" from="gnx" to="gnx"/>
    
Leo will use type="child" to represent Leo's official edges.
Plugins are free to define any type except "child". Examples::

    <edge type="undirected" from="gnx" to="gnx"/>
    <edge type="bidirectional" from="gnx" to="gnx"/>
    <edge type="backlink" from="gnx" to="gnx"/>
    <edge type="myPlugin" from="gnx" to="gnx"/>
    
Descendant attributes in @file trees
====================================

Descendants of @file nodes do not appear in .leo files. Thus,
important data must be stored in the so-called hidden machinery:
attributes of the @file node.

The <at-file-attributes> element may be contained in the
<node> element for @file nodes::

    <at-file-attributes>
        <attr>key="ua-name"
            format="(empty)/json/pickle/binhex"
            gnx="gnx">value
        </attr>
        ...
    </at-file-attributes>
    
In other words, we use the graphml <attr> element, extended with the
gnx attribute, to represent all the uA's in the descendants of @file nodes.
.. @+node:ekr.20110419083918.6104: *6* File format, v3 graphml
@language rest
@pagewidth 65

This draft is intended to reflect our recent discussions, with no
new invention from me. All comments and corrections welcome.

The draft is also intended to be compatible with graphml.

This draft covers only Leo's xml format, but it may be adapted
for use as a json file format.

I am willing to change "type" to "key" in <edge> elements if that
would be preferable.

Recent changes
==============

- Added <graphml> element defining the default namespace.

- Defined the leo namespace for leo-only elements.
    - Renamed <leo_file> to <leo:outline>
    - Renamed <descendant-attributes> to <leo:at-file-attributes>

- Used <leo:at-file-attributes> for marks, removing the special case.

- Enclosed <leo:descendant-attributes> in a (graphml) <data> element.

- Changed the format of the "marked" attribute to be a string-valued attribute.

First lines
===========

Leo file will start with the following::

    <?xml version="1.0" encoding="utf-8"?>
    <?xml-stylesheet my_stylesheet?>
    <!-- Created by Leo (http://webpages.charter.net/edreamleo/front.html) -->
    <leo:file file_format="3"
        xmlns:leo="http://www.leo-editor.org/2011/leo"/>
    <graphml xmlns="http://graphml.graphdrawing.org/xmlns"/>
        

No session data
===============

There will be no <globals>, <preferences> or
<find_panel_settings> elements. All such "session data" will be
restored from the cache, or from defaults if caching is disabled.

**Important**: there is no representation of expansion state or
the currently selected node anywhere in the .leo file.
Eliminating these data is contingent on having Leo work well with
caching disabled.

Note: marks are more than session data. They must appear
somewhere within <node> elements.


Summary of format
=================

.leo files will have the following form, with similar indentation::

    <?xml version="1.0" encoding="utf-8"?>
    <?xml-stylesheet my_stylesheet?>
    <!-- Created by Leo (http://webpages.charter.net/edreamleo/front.html) -->
    <leo:outline file_format="3" xmlns:leo="http://www.leo-editor.org/2011/leo"/>
    <graphml xmlns="http://graphml.graphdrawing.org/xmlns"/>
    <graph>
    <nodes>
        <!-- marked attribute appears only if the vnode is marked -->
        <node id="gnx"> 
            <head>headline text</head>
            <attr key="marked">1</attr>
            <!-- uA's... -->
            <!-- format="string" is the default -->
            <attr key="a">a string</attr>
            <attr key="b" format="json">a json string</attr>
            <attr key="c" format="pickle">a pickled string</attr>
            <attr key="d" format="binhex">a binhexed string</attr>
            ...
            <body>body text</body>
        </node>
        ...
        <!-- @file nodes contain a <data> package -->
        <node id="gnx">
            <head>@file x.py</head>
            ...
            <data>
                <leo:at-file-attributes>
                    <attr>key="ua-name"
                        format="(empty)/json/pickle/binhex"
                        gnx="gnx">value
                    </attr>
                    ...
                </leo:at-file-attributes>
            </data>
        </node>
        ...
    </nodes>
    <edges>
        <edge type="child" from="gnx" to="gnx"</edge>
        ...
    </edges>
    </graph>
    </graphml>
    </leo:outline>

<attr> elements
===============

<attr> elements will one of the following forms::

    <attr key="a">a unicode string</attr>
    <attr key="b" format="json">a json string</attr>
    <attr key="c" format="pickle">a json string</attr>
    <attr key="d" format="binhex">a binhexed string</attr>
    
That is, the value will be a string by default.

uA's that start with "binhex_" will use the binhex format. This
prefix must be retained in the type field, so the read code can
restore them.

If the value is not a string, and there is no "binhex_" prefix,
the write code will use format="json" if json.dumps succeeds, and
will use format="pickle" otherwise.

No <attr> element will be written if both json.dumps and
pickle.dumps fail. Attribute failures will create a warning for
the plugin developer.

<edge> elements
===============

<edge> elements will have the form::

    <edge type="child" from="gnx" to="gnx"/>
    
Leo will use type="child" to represent Leo's official edges.
Plugins are free to define any type except "child". Examples::

    <edge type="undirected" from="gnx" to="gnx"/>
    <edge type="bidirectional" from="gnx" to="gnx"/>
    <edge type="backlink" from="gnx" to="gnx"/>
    <edge type="myPlugin" from="gnx" to="gnx"/>
    
Descendant attributes in @file trees
====================================

Descendants of @file nodes do not appear in .leo files. Thus,
important data must be stored in the so-called hidden machinery:
attributes of the @file node.

The <leo:at-file-attributes> element may be contained in the
<node> element for @file nodes. For compatibility with graphml,
it will enclosed in a data element::
    
    <data>
        <leo:at-file-attributes>
            <attr>key="ua-name"
                format="(empty)/json/pickle/binhex"
                gnx="gnx">value
            </attr>
            ...
        </leo:at-file-attributes>
    </data>
    
In other words, we use the graphml <attr> element, extended with the
gnx attribute, to represent all the uA's in the descendants of @file nodes.
.. @+node:ekr.20090218115025.3: *6* Why are attributes pickled by default?
@nocolor-node

http://groups.google.com/group/leo-editor/browse_thread/thread/326a221f4c698f7a

> On Wed, Feb 18, 2009 at 12:12 PM, Kent Tenney <ktenney@gmail.com> wrote:
>>
>> Currently, Leo pickles the value of unknown attributes unless
>> the name starts with 'str_'
>>
>> Running the following code in node 'UA'
>>
>> p = c.currentPosition()
>> p.v.u = {'hello':'world', 'str_hello':'world'}
>>
>> results in the following in the .leo file:
>>
>> <v t="ktenney.20090218114928.367" str_hello="world"
>> hello="5505776f726c6471002e"><vh>UA</vh></v>
>>
>> I think this is surprising, Python-centric and contrary to the
>> spirit of Leo as a flexible data management platform.
>
> I suppose your point is that you can't create an arbitrarily named attribute
> with a string value. Does that create a real problem?

It requires a translation layer, either to (un)munge the name or
(un)pickle. Real problem?

Let's say each time I think 'I can use UAs to store that' I change
my mind when I realize my values will be in a pickle. (I really don't
want to name all my attributes str_xxx)

> As far as being Python-centric, can you suggest any other way of converting
> arbitrary data to a text string?

How is it done in any other XML file?
I've not used XML for arbitrary data, but it probably can be done.

> Why would that way be better than pickle?

My suspicion is that UAs would be used more for
storing text and numbers (as seems common for XML files)
than Python data objects.

Does Leo use UAs to store pickles?

I'm sure pickling capability is great, but I'm not convinced
it should be the _default._

No big deal.
.. @+node:ekr.20110415173840.6098: *5* Code related to uA's
.. @+node:ekr.20040701065235.2: *6* putDescendentAttributes
def putDescendentAttributes (self,p):

    nodeIndices = g.app.nodeIndices

    # Create lists of all tnodes whose vnodes are marked or expanded.
    marks = [] ; expanded = []
    for p in p.subtree():
        v = p.v
        if p.isMarked() and p.v not in marks:
            marks.append(v)
        if p.hasChildren() and p.isExpanded() and v not in expanded:
            expanded.append(v)

    result = []
    for theList,tag in ((marks,"marks"),(expanded,"expanded")):
        if theList:
            sList = []
            for v in theList:
                sList.append("%s," % nodeIndices.toString(v.fileIndex))
            s = ''.join(sList)
            # g.trace(tag,[str(p.h) for p in theList])
            result.append('\n%s="%s"' % (tag,s))

    return ''.join(result)
.. @+node:ekr.20080805071954.2: *6* putDescendentVnodeUas
def putDescendentVnodeUas (self,p):

    '''Return the a uA field for descendent vnode attributes,
    suitable for reconstituting uA's for anonymous vnodes.'''

    trace = False
    if trace: g.trace(p.h)

    # Create aList of tuples (p,v) having a valid unknownAttributes dict.
    # Create dictionary: keys are vnodes, values are corresonding archived positions.
    pDict = {} ; aList = []
    for p2 in p.self_and_subtree():
        if hasattr(p2.v,"unknownAttributes"):
            aList.append((p2.copy(),p2.v),)
            pDict[p2.v] = p2.archivedPosition(root_p=p)

    # Create aList of pairs (v,d) where d contains only pickleable entries.
    if aList: aList = self.createUaList(aList)
    if not aList: return ''

    # Create d, an enclosing dict to hold all the inner dicts.
    d = {}
    for v,d2 in aList:
        aList2 = [str(z) for z in pDict.get(v)]
        # g.trace(aList2)
        key = '.'.join(aList2)
        d[key]=d2

    if trace: g.trace(p.h,g.dictToString(d))

    # Pickle and hexlify d
    return d and self.pickle(
        torv=p.v,val=d,tag='descendentVnodeUnknownAttributes') or ''
.. @+node:EKR.20040526202501: *6* putUnknownAttributes
def putUnknownAttributes (self,torv):

    """Put pickleable values for all keys in torv.unknownAttributes dictionary."""

    attrDict = torv.unknownAttributes
    if type(attrDict) != type({}):
        g.es("ignoring non-dictionary unknownAttributes for",torv,color="blue")
        return ''
    else:
        val = ''.join([self.putUaHelper(torv,key,val) for key,val in attrDict.items()])
        # g.trace(torv,attrDict)
        return val
.. @+node:ekr.20090130114732.6: *6* v.u Property
def __get_u(self):
    v = self
    if not hasattr(v,'unknownAttributes'):
        v.unknownAttributes = {}
    return v.unknownAttributes

def __set_u(self,val):
    v = self
    if val is None:
        if hasattr(v,'unknownAttributes'):
            delattr(v,'unknownAttributes')
    elif type(val) == type({}):
        v.unknownAttributes = val
    else:
        raise ValueError

u = property(
    __get_u, __set_u,
    doc = "vnode unknownAttribute property")
.. @+node:ekr.20110528034751.18273: *4* Global search in Nav plugin?
.. @+node:ekr.20110528034751.18272: *4* Support for tabifying Leo's core panes
Collaborate with Terry.

A. Place separate body editors in free_layout areas. This should *easy* to
   do! Almost nothing changes in the code, but the visual effect should be
   much better.

B. Allow any pane to be "tabified" (placed in a tab in the Log pane) and
   "untabified." There are a few details to be handled, but nothing major.
   
C. Labels for panes.
.. @+node:ekr.20110613110911.16421: *4* Read/write files in json format
.. @+node:ekr.20110527225107.18351: *3* Vague
@language rest

**Important**: These items are not scheduled for any release. They will be done
only if there are specific requests for them.

Eventually, all these items will move to the dreaded to-do-later list.
.. @+node:ekr.20110529115328.18238: *4* Emacs related: 5
I'll do these if and and only if somebody asks for them.
.. @+node:ekr.20110529104352.18248: *5* Complete k.universalDispatche
.. @+node:ekr.20110529104352.18249: *5* Complete number-to-register command
.. @+node:ekr.20031218072017.753: *5* Emacs comint-mode
@nocolor

The improved Execute Script command does most of this

Michael Manti
mmanti@mac.com

P.S. I think a feature that could make Leo *the* IDE for developing in 
interpreted languages is something like the (X)Emacs comint-mode.el for 
interacting with the shell and interpreters.

comint-mode.el serves as the basis for interactive modes for a number of
languages--OCaml, Haskell, SML, among them. It allows for editing expressions in
one buffer and triggering their evaluation in another buffer that has an
interpreter running in it, along with entering commands in the interpreter
buffer and moving back and forth through the history of their evaluation.

Imagine being able to highlight a node in Leo, and have all the code in it and
its children evaluated in an interpreter running in a separate window or pane,
much as Leo can open a Python shell now. Users of those languages could build
plug-ins specific to their language atop that layer, and the @language directive
could activate that. I think that would be very cool.
.. @+node:ekr.20071004120359.2: *5* expand-region-abbrev
See: regionalExpandAbbrev.

You may wish to expand an abbrev with a prefix attached; for example, if `cnst'
expands into `construction', you might want to use it to enter `reconstruction'.
It does not work to type recnst, because that is not necessarily a defined
abbrev. What you can do is use the command M-' (abbrev-prefix-mark) in between
the prefix `re' and the abbrev `cnst'. First, insert `re'. Then type M-'; this
inserts a hyphen in the buffer to indicate that it has done its work. Then
insert the abbrev `cnst'; the buffer now contains `re-cnst'. Now insert a
non-word character to expand the abbrev `cnst' into `construction'. This
expansion step also deletes the hyphen that indicated M-' had been used. The
result is the desired `reconstruction'.

If you actually want the text of the abbrev in the buffer, rather than its
expansion, you can accomplish this by inserting the following punctuation with
C-q. Thus, foo C-q , leaves `foo,' in the buffer.
.. @+node:ekr.20060628103226.3: *5* Make sure repeat counts work on basic editing commands
.. @+node:ekr.20110527225107.18352: *4* Maybe: 5
.. @+node:ekr.20090724081340.5987: *5* Improve recursive import script and @auto
Instead of adding an @ignore directive, it might be better
to change @auto to @@auto.

Should @auto be more lenient with C files?

Improve the recursive import script.
    - Minimize the path names
    - Option to include/exclude the @auto itself

.. @+node:ekr.20051110155735.1: *5* Improve Spell tab & spell checker
@nocolor

- Per-pane key bindings. (arrows, etc.)
- Try default fonts for spell buttons.
- Select the first entry.

@color
.. @+node:ekr.20090907080624.6081: *6* Spell checker should check headlines
.. @+node:ekr.20101004092958.5914: *5* Write treepad scanner
@ treepad.py is from the treepad website
.. @+node:ekr.20101004092958.5939: *6* treepad.py
@first #! /usr/local/bin/python

# treepad.py

@language python
@tabwidth -4
@others
if __name__ == '__main__':
    Main().Run()

.. @+node:ekr.20101004092958.5940: *7* treepad declarations
import sys, os, re, string

# constants
VERSION = "<Treepad version 2.7>"

# regexes
END_RE = re.compile(r'^<end node> ([^ ]+)$')
.. @+node:ekr.20101004092958.5941: *7* class Node
class Node:
    @others
.. @+node:ekr.20101004092958.5942: *8* __init__
def __init__(self):
    self.title    = ""
    self.level    = 0
    self.article  = []
    self.children = []
    self.parent   = None
    self.end      = ""
.. @+node:ekr.20101004092958.5943: *8* __str__
def __str__(self):
    return "%s/%d" % (self.title, self.level)
.. @+node:ekr.20101004092958.5944: *8* addchild
def addchild(self, node):
    assert self.level == node.level-1 and node.parent is None
    node.parent = self
    self.children.append( node )
.. @+node:ekr.20101004092958.5945: *8* findparent
def findparent(self, node):
    if self.level == (node.level-1): return self
    return self.parent.findparent(node)
.. @+node:ekr.20101004092958.5946: *8* writenode
def writenode(self, fp):
    fp.write("dt=Text\n")
    fp.write("<node>\n")
    fp.write("%s\n" % self.title)
    fp.write("%s\n" % self.level)
    for line in self.article:
        fp.write("%s\n" % line)
    fp.write("<end node> %s\n" % self.end)
.. @+node:ekr.20101004092958.5947: *8* writetree
def writetree(self, fp):
    self.writenode(fp)
    for node in self.children:
        node.writetree(fp)

.. @+node:ekr.20101004092958.5948: *7* class NodeReader
class NodeReader:
    @others
.. @+node:ekr.20101004092958.5949: *8* __init__
def __init__(self, fname, fp):
    self.fname    = fname
    self.fp       = fp
.. @+node:ekr.20101004092958.5950: *8* expect
def expect(self, text, line=None):
    if line is None:
        line = self.fp.readline().strip()
    assert line == text, "expected " + line + " == " + text
.. @+node:ekr.20101004092958.5951: *8* readstart
def readstart(self):
    self.expect(VERSION)
.. @+node:ekr.20101004092958.5952: *8* readnode
def readnode(self):
    line = self.fp.readline()
    if line is None:
        return None
    line = line.strip()
    if len(line) < 1:
        return None
    self.expect("dt=Text", line)
    self.expect("<node>")
    node = Node()
    node.title = self.fp.readline().strip()
    node.level = int(self.fp.readline().strip())
    while 1:
        line = self.fp.readline()
        m = re.match(END_RE, line)
        if m:
            node.end = m.group(1).strip()
            break
        node.article.append( line.strip() )
    return node

.. @+node:ekr.20101004092958.5953: *7* class TreeReader
class TreeReader:
    @others
.. @+node:ekr.20101004092958.5954: *8* __init__
def __init__(self, fname, fp=None):
    if fp is None: fp = open(fname, 'r')
    self.nodereader = NodeReader(fname, fp)
    self.root = None
    self.prev = None
.. @+node:ekr.20101004092958.5955: *8* add
def add(self, node):
    if self.prev is None:
        assert node.level == 0
        self.root = node
    else:
        assert node.level > 0
        parent = self.prev.findparent(node)
        parent.addchild( node )
    self.prev = node
.. @+node:ekr.20101004092958.5956: *8* read
def read(self):
    self.nodereader.readstart()
    prev = None
    while 1:
        node = self.nodereader.readnode()
        if node is None: break
        self.add(node)

.. @+node:ekr.20101004092958.5957: *7* class TreeWriter
class TreeWriter:
    @others
.. @+node:ekr.20101004092958.5958: *8* __init__
def __init__(self, fname, fp=None):
    if fp is None: fp = open(fname, 'w')
    self.fname = fname
    self.fp    = fp
.. @+node:ekr.20101004092958.5959: *8* write
def write(self, root):
    self.fp.write("%s\n" % VERSION)
    root.writetree(self.fp)

.. @+node:ekr.20101004092958.5960: *7* class Main
class Main:
    @others
.. @+node:ekr.20101004092958.5961: *8* __init__
def __init__(self):
    self.infile  = sys.argv[1]
    self.outfile = sys.argv[2]
    self.reader  = TreeReader(self.infile)
    self.writer  = TreeWriter(self.outfile)
.. @+node:ekr.20101004092958.5962: *8* Run

def Run(self):
    self.reader.read()
    self.writer.write(self.reader.root)

.. @+node:ekr.20110520051220.18203: *5* Cross-tab search
@language rest

This would be a substitute for cross-file clones.
.. @+node:ekr.20110525112110.18402: *4* Ideas: 4
.. @+node:ekr.20071001052501: *5* Versioning for nodes
@nocolor

One feature I have not seen in SCS system is something which might be called
"history compression": I might be interested in having both version 5 and 6
in my source tree, when the current version is 7, but I am not really interested
in the 2000 steps which transformed 5 into 6 (just suggested this feature to
the bazaar people). This happens actually quite often to me, since I use the
SCS as a back-up system, saving many (uninteresting) intermediate steps while
implementing a new feature.
.. @+node:ekr.20101104191857.5820: *5* QWebView makes Leo a presentation tool
http://groups.google.com/group/leo-editor/browse_thread/thread/4ea2d3f7d2c68106#

Ville:

Create one QWebView window, zoom it in to have large fonts.

Create @button that converts current node containing restructuredtext to html,
and pushes that html to QWebView.

Voila', instant presentation tool. The webview window would be on projector, and
leo would be in your private computer. You can easily edit the text, or find new
interesting slides to present in privacy of your own screen.

.. @+node:ekr.20080802070659.11: *5* Make node attributes visible, and supported by Leo's core
.. @+node:ekr.20060227123536: *5* Tiddlywiki and related comments about rendering body text (Mulder)
@nocolor
http://sourceforge.net/forum/message.php?msg_id=3578252
By: bwmulder

I have been thinking for a while that it ought to be possible to somehow  to
unite Leo with wiki features (my thinking is still vague at this point).

If you look at systems like Tiddlywiki (http://www.tiddlywiki.com/) you will
find that they already pretty much provide all the formatting features mentioned
in the article.

MoinMoin, another wiki (http://moinmoin.wikiwikiweb.de), has started to use
a graphical interface for editing in the latest version.

Maybe Leo can be split up into three components:

1. A storage component is responsible for storing nodes. Currently, this is
just memory, but databases like shelve, Zope or sqllite should also be possible.

2. The control component is responsible for converting from the internal format
to external files which can be processed by existing compilers, searching within
a document, and the like.

3. A display component is responsible for interfacing with the user. If can
be TK, but it can also be something like the Tiddlywiki interface, which immediately
shows the formatting applied to text.

I don't know much about javascript, so I would have to learn more about this
language before doing anything in this direction.

As an intermediate step, maybe we could allow mixing RST processing with regular
program text.  Leo would produce two documents out of a source file: a version
for the compiler in plain ascii, and an HTML file for reading the source.
.. @+node:ekr.20110531190516.19365: *3* Minor/Maybe: 5
@language rest

.. @+node:ekr.20110614123640.6587: *4* Add headline/color functions to Leo's core
@nocolor-node

http://groups.google.com/group/leo-editor/browse_thread/thread/7e279fe3dedf42be/f00fde4df5b39ded

What uA should be used to specify node colors?

if the foreground / background color API uses uAs,
would/should the uAs use the reserved "leo_&lt;something&gt;"
namespace?

-------------------

Terry Brown

Sounds like something I may have brought up, long ago.

I was thinking that setting the fore/background color of nodes in the
tree should be a "gui core" function, and that the info should be
stored in uA, and so wanted to know what key should be used in uA for
that.  I think the docs say top level keys starting with "leo_" are
reserved, and probably wanted a ruling on

v.u['leo_fg'] = 'red'

vs

v.u['leo_tree_style']['fg'] = 'red'

etc.

I think the question may be more complicated than just what to call the
key, so you can probably retire the todo item.
.. @+node:ekr.20100112051224.6226: *4* Vim-related: Range prefix to commands/objects (k.getArgs)
The ability to specify a numeric range prefix is not supported. For example,
entering "3dd" will not delete the next three lines and "20G" will not move the
cursor to the 20th line in the file.

The ability to specify a numeric range prefix to an object is not supported. For
example, the "d2fx" command should Delete up to and including the 2nd Found "x"
character.
.. @+node:ekr.20110601105631.19373: *5* Simple vim bindings
.. @+node:ekr.20110601105631.19374: *6* Cursors
Vi normally uses two different "current character" designators depending on the
current state.

Insert state:

In the Insert state, a vertical bar is placed between two characters to indicate
where the next key will be inserted. Leo's cursor is of this type.

Command state: 

In the Command state, vi expects that the cursor is highlighting a current
character and provides commands to enter the insert state or paste text either
before or after that current character. Leo's vi emulation currently does not
support a "current character" cursor. As a result, inserting and pasting before
or after is replaced by inserting or pasting "at" the current cursor location.
For example, the 'i' and 'a' command are both mapped to enter the insert state
at the current cursor location.
.. @+node:ekr.20110601105631.19375: *6* Enter insert mode after ctrl-h
.. @+node:ekr.20110601105631.19376: *6* Colorize headline text depending on state
.. @+node:ekr.20110601105631.19377: *6* colon destroys alt-x binding
This project reorganizes makeBindingFromCommandsDict
.. @+node:ekr.20110530063322.18333: *4* scala not colored properly
.. @+node:ekr.20110518103946.18179: *4* Add external/leosax.py to leoPyRef.leo
http://groups.google.com/group/leo-editor/browse_thread/thread/93f2cc88ebbf9893

Would be ok with you if I pulled it into leoPy.leo, thereby adding sentinels to it? 
.. @+node:ekr.20090801103907.6018: *4* Add entries to global dicts for more languages (waiting for requests)
http://groups.google.com/group/leo-editor/browse_thread/thread/b41ddfeb3c84e780

Especially languages in leo/modes.

** Only c.getOpenWithExt uses app.language_extension_dict.

I'll wait until I get requests for particular language.
.. @+node:ekr.20110528103005.18323: *5* Found: extension_dict
.. @+node:ekr.20080819075811.13: *6* adjustTargetLanguage
def adjustTargetLanguage (self,fn):

    """Use the language implied by fn's extension if
    there is a conflict between it and c.target_language."""

    at = self ; c = at.c

    if c.target_language:
        junk,target_ext = g.os_path_splitext(fn)  
    else:
        target_ext = ''

    junk,ext = g.os_path_splitext(fn)

    if ext:
        if ext.startswith('.'): ext = ext[1:]

        language = g.app.extension_dict.get(ext)
        if language:
            c.target_language = language
        else:
            # An unknown language.
            pass # Use the default language, **not** 'unknown_language'
.. @+node:ekr.20090212054250.9: *6* c.createNodeFromExternalFile
def createNodeFromExternalFile(self,fn):

    '''Read the file into a node.
    Return None, indicating that c.open should set focus.'''

    c = self

    s,e = g.readFileIntoString(fn)
    if s is None: return
    head,ext = g.os_path_splitext(fn)
    if ext.startswith('.'): ext = ext[1:]
    language = g.app.extension_dict.get(ext)
    if language:
        prefix = '@color\n@language %s\n\n' % language
    else:
        prefix = '@killcolor\n\n'
    p2 = c.insertHeadline(op_name='Open File', as_child=False)
    p2.h = '@edit %s' % fn # g.shortFileName(fn)
    p2.b = prefix + s
    w = c.frame.body.bodyCtrl
    if w: w.setInsertPoint(0)
    c.redraw()
    c.recolor()
.. @+node:ekr.20031218072017.2824: *6* c.getOpenWithExt
def getOpenWithExt (self,p,ext):

    trace = False and not g.app.unitTesting
    c = self

    if not ext:
        # if node is part of @<file> tree, get ext from file name
        for p2 in p.self_and_parents():
            if p2.isAnyAtFileNode():
                fn = p2.h.split(None,1)[1]
                ext = g.os_path_splitext(fn)[1]
                if trace: g.trace('found node:',ext,p2.h)
                break

    if not ext:
        theDict = c.scanAllDirectives()
        language = theDict.get('language')
        ext = g.app.language_extension_dict.get(language)
        if trace: g.trace('found directive',language,ext)

    if not ext:
        ext = '.txt'
        if trace: g.trace('use default (.txt)')

    if ext[0] != '.':
        ext = '.'+ext

    return ext
.. @+node:EKR.20040504150046.4: *6* g.comment_delims_from_extension
def comment_delims_from_extension(filename):

    """
    Return the comment delims corresponding to the filename's extension.

    >>> import leo.core.leoGlobals as g
    >>> g.comment_delims_from_extension(".py")
    ('#', '', '')

    >>> g.comment_delims_from_extension(".c")
    ('//', '/*', '*/')

    >>> g.comment_delims_from_extension(".html")
    ('', '<!--', '-->')

    """

    if filename.startswith('.'):
        # Python 2.6 changes how splitext works.
        root,ext = None,filename
    else:
        root, ext = os.path.splitext(filename)
    if ext == '.tmp':
        root, ext = os.path.splitext(root)

    language = g.app.extension_dict.get(ext[1:])
    if ext:
        return g.set_delims_from_language(language)
    else:
        g.trace("unknown extension: %s, filename: %s, root: %s" % (
            repr(ext),repr(filename),repr(root)))
        return '','',''
.. @+node:ekr.20080811174246.1: *6* languageForExtension
def languageForExtension (self,ext):

    '''Return the language corresponding to the extension ext.'''

    unknown = 'unknown_language'

    if ext.startswith('.'): ext = ext[1:]

    if ext:
        z = g.app.extra_extension_dict.get(ext)
        if z not in (None,'none','None'):
            language = z
        else:
            language = g.app.extension_dict.get(ext)
        if language in (None,'none','None'):
            language = unknown
    else:
        language = unknown

    # g.trace(ext,repr(language))

    # Return the language even if there is no colorizer mode for it.
    return language
.. @+node:ekr.20031218072017.368: *5* << define global data structures >> (leoApp.py)
# Internally, lower case is used for all language names.
self.language_delims_dict = {
    # Keys are languages, values are 1,2 or 3-tuples of delims.
    "ada"           : "--",
    "batch"         : "REM_", # Use the REM hack.
    "actionscript"  : "// /* */", #jason 2003-07-03
    "autohotkey"    : "; /* */", #TL - AutoHotkey language
    "c"             : "// /* */", # C, C++ or objective C.
    "config"        : "#", # Leo 4.5.1
    "csharp"        : "// /* */", # C#
    "cpp"           : "// /* */",# C++.
    "css"           : "/* */", # 4/1/04
    "cweb"          : "@q@ @>", # Use the "cweb hack"
    "cython"        : "#",
    "elisp"         : ";",
    "forth"         : "\\_ _(_ _)", # Use the "REM hack"
    "fortran"       : "C",
    "fortran90"     : "!",
    "haskell"       : "--_ {-_ _-}",
    "haxe"          : "//",
    "html"          : "<!-- -->",
    "ini"           : ";",
    "java"          : "// /* */",
    "kshell"        : "#", # Leo 4.5.1.
    "latex"         : "%",
    "lisp"          : ";", # EKR: 2010/09/29
    "lua"           : "--",  # ddm 13/02/06
    "nsi"           : ";", # EKR: 2010/10/27
    "noweb"         : "%", # EKR: 2009-01-30. Use Latex for doc chunks.
    "pascal"        : "// { }",
    "perl"          : "#",
    "perlpod"       : "# __=pod__ __=cut__", # 9/25/02: The perlpod hack.
    "php"           : "// /* */", # 6/23/07: was "//",
    "plain"         : "#", # We must pick something.
    "plsql"         : "-- /* */", # SQL scripts qt02537 2005-05-27
    "python"        : "#",
    "rapidq"        : "'", # fil 2004-march-11
    "rebol"         : ";",  # jason 2003-07-03
    "rest"          : ".._",
    "rst"           : ".._",
    "ruby"          : "#",  # thyrsus 2008-11-05
    "scala"         : "// /* */",
    "shell"         : "#",  # shell scripts
    "tcltk"         : "#",
    "tex"           : "%", # Bug fix: 2008-1-30: Fixed Mark Edginton's bug.
    "unknown"       : "#", # Set when @comment is seen.
    "unknown_language" : '#--unknown-language--',
        # For unknown extensions in @shadow files.
    "vim"           : "\"",
    "vimoutline"    : "#",  #TL 8/25/08 Vim's outline plugin
    "xml"           : "<!-- -->",
    "xslt"          : "<!-- -->",
}

# Used only by c.getOpenWithExt.
self.language_extension_dict = {
    # Keys are languages, values are extensions.
    "ada"           : "ada",
    "actionscript"  : "as", #jason 2003-07-03
    "autohotkey"    : "ahk", #TL - AutoHotkey language
    "batch"         : "bat", # Leo 4.5.1.
    "c"             : "c",
    "config"        : "cfg",
    "cpp"           : "cpp",
    "css"           : "css", # 4/1/04
    "cweb"          : "w",
    #"cython"        : "pyd",
    #"cython"        : "pyi",
    "cython"        : "pyx", # Only one extension is valid at present.
    "elisp"         : "el",
    "forth"         : "forth",
    "fortran"       : "f",
    "fortran90"     : "f90",
    "haskell"       : "hs",
    "haxe"          : "hx",
    "html"          : "html",
    "ini"           : "ini",
    "java"          : "java",
    "kshell"        : "ksh", # Leo 4.5.1.
    "latex"         : "tex", # 1/8/04
    "lua"           : "lua",  # ddm 13/02/06
    "nsi"           : "nsi", # EKR: 2010/10/27
    "noweb"         : "nw",
    "pascal"        : "p",
    "perl"          : "pl",      # 11/7/05
    "perlpod"       : "pod",  # 11/7/05
    "php"           : "php",
    "plain"         : "txt",
    "python"        : "py",
    "plsql"         : "sql", # qt02537 2005-05-27
    "rapidq"        : "bas", # fil 2004-march-11
    "rebol"         : "r",    # jason 2003-07-03
    # "rst"           : "rst", # caught by pylint.
    "rst"           : "rest",
    "ruby"          : "rb",   # thyrsus 2008-11-05
    "scala"         : "scala",
    "shell"         : "sh",   # DS 4/1/04
    "tex"           : "tex",
    "tcltk"         : "tcl",
    "unknown"       : "txt", # Set when @comment is seen.
    "vim"           : "vim",
    "vimoutline"    : "otl",  #TL 8/25/08 Vim's outline plugin
    "xml"           : "xml",
    "xslt"          : "xsl",
}

self.extension_dict = {
    # Keys are extensions, values are languages.
    "ada"   : "ada",
    "adb"   : "ada",
    "ahk"   : "autohotkey",  # EKR: 2009-01-30.
    "as"    : "actionscript",
    "bas"   : "rapidq",
    "bat"   : "batch",
    "c"     : "c",
    "cfg"   : "config",
    "cpp"   : "cpp",
    "css"   : "css",
    "el"    : "elisp",
    "forth" : "forth",
    "f"     : "fortran",
    "f90"   : "fortran90",
    "h"     : "c",
    "html"  : "html",
    "hs"    : "haskell",
    "ini"   : "ini",
    "java"  : "java",
    "ksh"   : "kshell", # Leo 4.5.1.
    "lua"   : "lua",  # ddm 13/02/06
    "nsi"   : "nsi", # EKR: 2010/10/27
    "nw"    : "noweb",
    "otl"   : "vimoutline",  #TL 8/25/08 Vim's outline plugin
    "p"     : "pascal",
    "pl"    : "perl",   # 11/7/05
    "pod"   : "perlpod", # 11/7/05
    "php"   : "php",
    "py"    : "python",
    "pyd"   : "cython",
    "pyi"   : "cython",
    "pyx"   : "cython",
    "sql"   : "plsql", # qt02537 2005-05-27
    "r"     : "rebol",
    "rb"    : "ruby", # thyrsus 2008-11-05
    "rest"  : "rst",
    "rst"   : "rst",
    "scala" : "scala",
    "sh"    : "shell",
    "tex"   : "tex",
    "txt"   : "plain",
    "tcl"   : "tcltk",
    "vim"   : "vim",
    "w"     : "cweb",
    "xml"   : "xml",
    "xsl"   : "xslt",
    "hx"    : "haxe",
}

# Extra language extensions, used to associate extensions with mode files.
# Used by importCommands.languageForExtension.
# Keys are extensions, values are corresponding mode file (without .py)
# A value of 'none' is a signal to unit tests that no extension file exists.
self.extra_extension_dict = {
    'actionscript': 'actionscript',
    'ada'   : 'ada95',
    'adb'   : 'none', # ada??
    'awk'   : 'awk',
    'bas'   : 'none', # rapidq
    'bat'   : 'none', # batch
    'cfg'   : 'none', # Leo 4.5.1
    'cpp'   : 'c',
    'el'    : 'lisp',
    'f'     : 'fortran90',
    'hx'    : 'none',
    'ksh'   : 'none', # Leo 4.5.1
    'nsi'   : 'none', # Leo 4.8.
    'nw'    : 'none', # noweb.
    'otl'   : 'none', # vimoutline.
    'pod'   : 'perl',
    'tcl'   : 'tcl',
    'unknown_language': 'none',
    'w'     : 'none', # cweb
}

self.global_commands_dict = {}
.. @+node:ekr.20110528103005.18319: *5* Script to global data structures from in modes/*.py files
import glob
import imp

theDir = g.os_path_finalize_join(g.app.loadDir,'..','modes','*.py')
aList = glob.glob(theDir)
theDir = g.os_path_finalize_join(g.app.loadDir,'..','modes')

print('-'*40,len(aList))
known_keys = list(g.app.language_delims_dict.keys())
known_keys.sort()
known = []
computed = {}
for z in aList:
    name = g.os_path_basename(z)
    name2 = name[:-3]
    if name2.startswith('__'): continue
    # if name2 in known_keys:
        # known.append(name)
        # continue
    try:
        theFile, pathname, description = imp.find_module(name2,[theDir])
        m = imp.load_module(name2, theFile, pathname, description)
    except Exception:
        g.es_exception()
        m = None
    if m:
        aList2 = [m.properties.get(z)
            for z in ('lineComment','commentStart','commentEnd')
                if m.properties.get(z)]
        print('%-20s : "%s",' % (
            '"%s"' % (name2),
            ' '.join(aList2)))
        computed[name2] = ' '.join(aList2)
mismatches = 0
for z in known_keys:
    val = g.app.language_delims_dict.get(z)
    val2 = computed.get(z)
    if not val:
        print('oops: no val',z)
    elif not val2:
        print('oops: no val2',z)
    elif val != val2:
        mismatches += 1
        print('mismatch for %s' % z)
        print(repr(val))
        print(repr(val2))
print('%s total languages' % len(aList))
print('%s new languages' % (len(list(computed.keys())) - len(known_keys)))
print('%s mismatches' % mismatches)
print('%s known language: %s' % (len(known_keys),known_keys))
.. @-all
.. @-leo
