ARB change log Major changes for next release: - ARB PARSIMONY * topology optimization - now (by default) strictly restricted to marked/visible parts of the tree (#640) - restriction now customizable (marked/all; visible/all) - tree costs for protein-data were not independent from root-position (as expected by model; #633). Caused infinite running optimization under some circumstances. - optimize-modes now strictly restrict to clicked subtrees. single/repeated optimization possible. - KL-optimizer * static path reduction slightly changed meaning. changed default settings. * removed randomness (was just covering some bugs) * improved general optimization speed * branchlength calculation - "forgot" to recalculate lengths under some conditions. fixed. - is now independent of tree-root position (#641) * adding species - 'add partial species' failed if two partial species had NO overlap (#609). fixed. - in 'add species + NNI' local optimization quality depended on insert position. fixed. - insertion of multiple species is now done independently (=unordered) - performance improved (esp. for many added species/big trees) * generally improved combine performance (using SSE) * generally reduced the number of performed combines (skipping many useless) * added function to randomize (parts of) the tree * warns about insufficient sequence data (e.g. as result of too restrictive filtering; #631) * fixed 'RESTORE' (crashed after deleting species from tree; #528) * corrected handling of dots ('.') while combining ancestor sequences * fixed a bunch of internal bugs (#620, #627, ...) * added species-info mode - DNA realigner * several unjustified failures will no longer happen (fixes #419 and most likely #145) - correctly re-syncs after 'X' (if possible at all) - no longer fails for 'B' and 'Z' - accepts 3 or more consecutive IUPAC codes in DNA * added option to cut-off DNA sequence (was done at end of sequence by old version. fixed) * fixed several minor bugs - ARB probeSpec: visualisation of probe set specificity (thanks to Paavo Jumppanen, CSIRO) - species selections (editor configurations): * visualisation of multiple selections in standard tree view (#658; example in database demo.arb) * order can be changed; each configuration has a comment - sequence import/export: * manage/edit/test import-/export-filter-definitions from inside ARB (#691) * import can store configuration of imported species (#607) * corrected EMBL export filter (numbers at seq.data) - Tree shading (#443) * according to values stored in database * according to given topology (useful when comparing topologies) - support for extra database compression (using gzip, bzip2, xz); databases cannot be opened by arb versions before 6.1.x - ARB_EDIT4: * display selected database fields as flags (allowing to toggle their value; #261). Example use: easily mark sequence as "curated" after manually checking its alignment. * allow to load missing SAIs * "view differences" to a reference sequence: - customizable: * char used for "equality" (i.e. what is displayed where a sequence is equal to selected sequence) * case-sensitivity * ignore different gap-types - equal data also gets hidden in consensus - refresh differences of all displayed sequences, when data of selected sequences changes - change reference sequence using CTRL-R or automatically let it follow the cursor - added hotkey to toggle mode: CTRL-D - fixed minor bugs * consensus calculation in ARB_EDIT4 and calculation of CONSENSUS SAI - now both calculations are strictly consistent: * gaps are now ignored while deciding whether to simplify using IUPAC ambiguity codes * IUPAC ambiguity codes encountered in sequence data are now counted proportionally (=> fewer 'N's occur in consensus) - added sliders to consensus definition windows - user defined consensus settings exchangable between both consensus setups - fixed and updated documentation * added species-info mode + database save * predefined SAI color translation for PVP - changes to SAI generation * MAX_FREQUENCY: - considers IUPAC ambiguity codes proportionally - amino acids: if MAX_FREQUENCY is below 10% SAI now shows '1' (prev. it did show '0', i.e. 100%) * POS_VAR_BY_PARSIMONY (PVP): - now (again) works with amino acid data - expand zombies in tree (unfold groups) - compare taxonomy (and mark differences; #651) - search&query for taxonomic groups (#652) - many search criteria (name, size, marked, nesting-level, ingroup-distance, ...) - search multiple trees, detect duplicate and missing groups - operations on found groups (delete, rename, fold, mark) - added concept of "inverse groups" (aka "keeled groups") - group transfer between trees: - penalties can be customized in detail - quality reports (to log and optionally to target name) - external (command line) aligners: * fixed incorrect handling of 'T' vs 'U': now all aligned sequences will contain the correct base depending on alignment type * preserve gap-type ('-' vs '.') and upper-/lower-case of original alignment * no longer ask what to do with aligned sequence, just overwrite it - only warn about real sequence changes (so please do NOT ignore from now on!) - config-managers: * possibility to restore factory defaults * added comment field for configurations * added them throughout arb (#647) - added slide controls throughout arb (#656) - tree (display) options: * fine grained scaling * add threshold for visible support values * group display (shading, customizable counters, triangle clades, optimized name+bootstrap display position) * diagonal branch style * parent branch position * all options are now also supported by ARB_PARSIMONY * improved auto-jump; now also works for groups * added optional auto-unfolding (to selected group/species) * select group on fold/unfold/create/move/.. * draw selected group in cursor-color * added keys for tree-traversal (moving selected species or group) - synchronized tree scrolling (#683) - colorsets were invalidated by generating new IDs (#660). fixed. - added alternate RAxML (DNA only; version 8.2.8) - multicore support (automatically activates recommended number of threads) - evaluation, optimization and extension of existing trees with RAxML - fix performance of "format sequences" (broken in arb-6.0.x series; #702) - ACI - added boolean operators, numeric comparisons, floating point arithmetic and several other new commands - improved ACI debugging: more verbose tracing (console log accessible from inside ARB) - NDS optionally uses only visible definitions - Search&Query (sort results numerically, improved macro compatibility) - Species info (improved detachment, field selection + macro compatibility) - updated integrated documentation Fixes for arb-6.0.5 (4 May 2016): - fixes for ubuntu 16.04 build Fixes for arb-6.0.4 (2 May 2016): - fixes for OSX build (SIP, accepted compilers) Fixes for arb-6.0.3 (19 Nov 2015): - fixes permission problems when multiple users share databases or ptservers (thx to Alan McCulloch) Fixes for arb-6.0.2 (8 Aug 2014): - compile issues on Snow Leopard (OSX 10.6) - merge Debian security fix for CVE-2008-5378 - small changes to build system for Debian - add desktop integration files Fixes for arb-6.0.1 (22 Jul 2014): - arb_parsimony - skip unwanted automatic branchlength recalculations (e.g. by unfolding a group) - corrected branchlength calculation for "Add marked partial species" - dots were treated as gaps for protein sequences (now treated as 'X'; analog to DNA treating gaps as 'N'). thx to Yan Shi for detecting that problem! - print - preview failed (showed empty postscript file) - print to file now always saves in user home - raxml (import tree with bootstrap values) Major changes for arb-6.0 (4 Jun 2014): - merge databases allows to - merge from an existing database into the database loaded in ARB_NT - merge to existing databases from the database loaded in ARB_NT - ARB can now - be restarted with another database and - a second instance of ARB can be opened - ARB_DIST - Detect clusters of species with similar sequences (OTUs) - allow automatic recalculation of matrix and/or tree whenever some parameter or data changes (only makes sense for smaller species sets) - extract distance matrix from tree - Rewrote chimera check. Allows filtering - added RNACMA (computes clusters of correlated positions) - PT-Server - changed behavior - no longer report less hits for a part of a probe than for the probe itself (occurred at 3'-end of alignment) - reports previously missing hits in joined genes - reports more hits at 3'-end of alignment (when using mismatches the PT-server now reports possible matches that go beyond the end of the sequence) - dots in the middle of the alignment act like the sequence ends there - minimum probe length reduced to 2 (was 6) - allow up to 50% of probe to mismatch - performance - optimized memory-estimation (will build in fewer passes) - uses any number of passes (not only 1, 5, 25, ...) - allows to define used memory by setting environment variable ARB_MEMORY - reduced memory needed to build/run ptserver (approx. 50%) - reduced size of indexfile (.pt) to ~50% - fast startup of existing ptservers - probe design - faster in many cases - allow to design probes of length 8 (previously 10) - allow to design probes with different lengths (specifying min/max length) - fixed number of outgroup hits reported when decreasing temperature (now each outgroup member only occurs once) - show possible reasons why no probes could be designed - probe match (allow any number of mismatches) - next relative search - can be restricted to column ranges (needs a PT-Server calculated from aligned sequences) - corrected and improved scaling of relative scores - more accurate scores (due to fixes in PT-Server; see below) - faster in many cases - show errors from ptserver build in ARB - fast-aligner - searches next-relatives based on selected column-block - align multiple column-blocks based on SAI - Rewrote alignment adaption during merge - Insert/delete columns using a SAI to define affected columns - ARB_EDIT4 - improved support for using multiple edit-windows - smoother refreshes - tweaked ORF display - tree importer/exporter - ARBs extended newick format (with bootstrap values) handled more restrictive now - fixed several bugs; improved errors/warnings - consensus trees - calculate from multiple existing trees (also allows to merge not completely overlapping trees) - fixed NJ-bootstrapping (no longer drops species) - tree display - Show brackets on open groups (dendrogram tree only) - rewrote IRS (folded) display - fixed tree key-bindings (mark, fold, ...) - improved several tree-commands (move, rotate, spread, length, width) - added a branch analysis tool - groups several functions previously available via menuitems (e.g. mark long branches, etc.) - added leaf-distance analysis - other tree functionality - treelist sortable now - new beautify-tree modes (radial tree / according to other tree) - function to remove marked/zombies from ALL trees - create multifurcations (by branchlength/bootstrap limit) - toggle 100% bootstrap values - tweaked printing (interface, overlapping) - if YOU edit a helpfile it will be automatically packed into an archive ready to be sent to ARB developers - probe design: - added LOAD to result window - automation - macro recording works in ARB client applications (ARB_EDIT4, ARB_PARS, ARB_MERGE, ..) - arb_ntree can execute macro from command line - added "Never ask again" to modal question boxes (for better compatibility with macros) - a macro can be called for all marked species (once for each) - macros can be nested (i.e. can call other macros) - support for user-specific customization: - of GDE menus (in ~/.arb_prop/gde) - of import/export filters (in ~/.arb_prop/filter) - ACI (some new commands, bugfixes) - updated/added external tools: - added FastTree (version 2.1.7) - added MAFFT (version 7.055) - added MrBayes (version 3.2.1) - added MUSCLE (version 3.8.31) - added PHYML (2013/07/08; also kept old version 2.4.5) - added PROBCONS (version 1.12) - updated RAxML (version 7.7.2) - load/save for window specific settings (e.g. allows to share parts of configuration with other users) - Support for mouse-wheel - many unlisted bugfixes - many internal refactorings Fixes for arb_5.5 (15 Nov 2012): * arb_5.4 was broken (several external tools missing) Fixes for arb_5.4 (14 Nov 2012): * make it obvious when probe matches are truncated. Truncate all hits beyond 1 million (was 100000) * fixed realigner (better interaction with fields 'transl_table' and 'codon_start'; improved error handling) * fixed several compilation issues (OSX; recent distro releases) Fixes for arb_5.3 (10 Nov 2011): - bugfixes - fixed wrong absolute/ecoli position reported for some designed probes - decompression error handling (pt-server build issues) - fixed 'codon_start' generated with wrong type - fixed a buffer overflow in ACI - report failures to write to /tmp - changes - markSpecies.pl: mark by accession number partial/ambiguous matches - internal fixes - compilation fixes for OSX - some patches for debian version (removed refs to xview, textedit, removed molphy(protml)) - removed obsolete dependency from libXp Fixes for arb_5.2 (5 Sep 2010): - bugfixes - quicksave did silently do nothing (especially not save anything) if an error occurred - ARB_EDIT4: crashed when using config with MANY unknown species - ARB_SECEDIT: crashed when trying to paint strand w/o any base - ARB_NTREE/ARB_PARS: crashed when clicking on inner tree node w/o groupinfo - changes - ARB uses xdg-open to display web-pages - internal fixes - karmic koala (gcc 4.4.1) - installation script - arb build process uses xsltproc instead of sablotron Fixes for arb_5.1 (1 Oct 2009): - fixed a bug in 'Create species from consensus' (created sequence was corrupted) - fixed 2 bugs in optimize DB (alignment w/o data, missing transaction) - updated installation instructions, fixed install script, added OSX instruction (thx to Matt Cottrell) - fixed broken demo.arb Major changes for arb_5.00 (4 Sep 2009): - ARB 64bit version - new genome importer - search for next relatives improved (normal search and fast-aligner) - new parameters to precise search - improved speed - partial sequence reach normal scores - search&query - supports regular expressions and ACI - track hit information - result sorting - Nameservers with add.field have to be started with default value You need to correct parameter -f in lib/arb_tcp.dat (according to lib/arb_tcp_org.dat) - multiple PT-servers may be used in parallel - fixed multiprobe - type-conversion for DB fields - SILVA compatible import filters - Newick tree export: - optionally save in human-readable format (big) - closer to newick standard format (quoting style, comment, special chars in data) - Upgraded RAxML to 7.0.3 (many features now usable from ARB interface) - Fixed sequence quality calculation - Secondary structures for proteins (DSSP) - Distance matrix (arb_dist): mark by distance to selected - ARB core - many bugfixes and improvements to reliability - faster sorting (general speedup) - improved sequence compression (avoid worse trees, better ratio) - improved handling of temporary files (permission/removal) - prints backtraces in userland - regular expression are POSIX standard now - macro record/playback - fixed several bugs - you need to re-record your old macros! - GUI: - disabled auto-focus, you need to click now - auto-raise windows on access - Minor things: - Ubuntu: packet installation for ARB - Fixed novice/expert mode - Mark deep/degenerated branches - Increased NDS entries - up-to-date Mac port (thx to Matt Cottrell) Major changes in ARB 07.12.07org (7 Dec 2007): - rewrote secondary structure editor - Sequence quality check - Nameserver may use one field additional to 'acc' (useful to keep multiple species with same acc) - tweaked base frequency filter generation - Normal export (not using readseq) improved: - supports filters and gap removal - optimized for big amount of data - reworked export filters - Display translation with different ORFs in EDIT4 - ARB exports in FIG 3.2 format (optionally in colors). Thanks to Elmar Pruesse. - added PHYML 2.4.5 (thanks to Stephane Guidon for the permission to distribute that great tool) - more compact display in EDIT4 - capable to use iso10646 fonts - supports various gcc versions (2.95.3 - 4.1.1) - fixed a bug in DB optimization (occurred when fields had bigger protection than current) - Bootstrap circles may be displayed as ellipses; upper size limit configurable; uses different color for size-limited circles; fixed xfig-export-bug - Allows Branchlength <-> Bootstrap value transfer (lossy!) - fixed several scaling bugs in "folded tree"-mode - improved import-filter error-messages - NDS-display of groups (e.g. in tree) is now handled by ACI-command 'taxonomy'. This gives several new possibilities: - export taxonomy via 'Export NDS list' - display taxonomy in Editor etc. - display of cascaded taxonomies - display taxonomy of tree_1 in tree_2 - allows to write taxonomy into database field of species - compare taxonomies of two trees - ... - ACI: - many new ACI commands - unified handling of binary ACI-operators - tracing of ACI actions for debugging purpose - ARB Neighbour joining: - bootstrap limit configurable - bugfix: when aborting bootstrap calculation, sometimes no tree was generated - EDIT4: - added unalign right (block-op) - added 'Save loaded properties' - GENE MAP: - multiple views possible at the same time - origin now at "12 o'clock" - implemented 'jump to gene' - tweaked file selection - Enhanced Search Depth for Probe Match --> max 20 MM - CLUSTALW: - separated menus for fast and slow alignment - most parameters accessible from inside ARB now - upgraded to PHYLIP 3.6 (adds PROML) - external programs may be called parallel (e.g. several treeing programs) - fixed bugs in protml and integration of protml - rewrote ASCII database import - arb_repair for databases of any size (script for database repair) - fixed bug in data compression - increased internal cache size (alignments up to 400.000bp possible w/o performance collapse) - ARBparsimony: increase hardcoded species limit (50.000 -> 250.000) - GDE menus cleanup - translation/re-alignment tweaked - unalign right (EDIT4) - visualization of SAIs in Probe Match Results - changed formatting of probe match results; increase # of allowed matches to 100.000; warn if results are truncated - PT server for genes - Probe design performance optimized - fixed NEXUS export format - exports group names into Newick format - import XML tree files - help for external tools now properly shown inside ARB Major changes in Beta 2003_08_22 (22 Aug 2003): - automatic formatting of alignments - SECEDIT may use EDIT4 colors - fixed bootstrapping (DNAPARS, PROTPARS, PROTML(experimental!)) - updated clustalw to version 1.83 - Restore window sizes for ALL windows (too small sizes are ignored) - new algorithm to add partial sequences to an existing tree - PROT-parsimony was completely redesigned and works now most similar to DNA/RNA-parsimony - Top area of ARB_NTREE may be reduced to maximize display area - All arb menus may be detached (click dashed line at top of menu) - visualization of SAIs (as background color behind Sequences) - ARB_EDIT4 may save/use alignment-specific and alignment-type-specific properties - PT-server occupies more memory => does less passes; more diagnostic output - small changes to status window (unhide behavior/time estimation) - menus and menu-hotkeys reorganized - colored buttons in color config windows - alignment concatenation (e.g. several different genes) - merging data of similar species (according selected database field) - keyboard commands for ARB_NTREE (mark/unmark/invert, collapse/expand) - expanded sellists - save/load fixed for multi probes - Binary SAIs are editable in ARB_EDIT4 - Information windows are detachable (allows to have multiple windows showing different items) - Scanning for hidden/unknown database fields improved and separated; possibility to remove unused fields. - new tabbed format in 'Export NDS' and 'Export matrix' (useful for star-calc/excel/etc.) - updated fastDNAml to 1.2.2 - added AxML (accelerated fastDNAml 1.2.2) - Field transfer definitions for exporting gene-species - File Selection: - recursive search available - The ARB_NTREE macro recording/execution has been fixed - Colorize species (see demo.arb) - Fixed missing-character-bug in Xfig, Print and Edit4-Info-Display - 'IslandHopper' -- a new integrated aligner (beta) - Many improvements and bugfixes to secondary structure editor: - highlighting of search (i.e for probes) like in EDIT4 - interactive constraint editing (stretch/compress) - probe info - editing secondary structure in XFIG now possible - visualization of SAIs - import reads Unix, DOS, and MAC linefeeds - NTREE/SAI/Etc/GnuPlot: calls gnuplot directly; more plotting features; basic help - tree and sequence export to XML ( DTDs are provided in ./lib/dtd ) (reloading of these XML files is planned for the future) - fixed problems with phylip-tree import/export (bootstrap values,comments,...) - search in all database fields possible ('[all fields]') - up to 10 quicksaves are kept - new ACI functions: upper, lower, caps, eval - variables for import filter programming - extract gene-species: creates acc; extraction to existing alignments - sequence of selected gene is mirrored in ARB_EDIT4/local_signature (=> selected gene can be highlighted in primary editor) - PCR primer-design for single genes - when selecting a gene, the corresponding gene-species is selected (if found) - save configuration for several windows (e.g. Search&Query, WWW, NDS, ...) - file selection box in import window - mark item with double click works in all search&query windows - User masks: create new; 'edit enable' and 'marked' toggles (like in info window) - Fixed command line help for all Arb-modules - Fixed problem parsing fonts (should fix display problems with default fonts) - Mark mode now works in list-view as well (ARB_NTREE) - Fixed appearance of 'tiny little boxes' (everywhere) - Redesign of ARB help: - a HTML version is in $ARBHOME/lib/help_html - a text version is in $ARBHOME/lib/help (like before, but now generated) Major changes in Beta 2001_11_07 (7 Nov 2001): - design probes to maximum length of 60 nucleotides - fastAligner1.03 bug fixed (chooses best match now in 'auto search' mode) - import default changed to foreign data format, ali name '16s' - printing of multi-page-trees works again - implemented user defineable masks to access database fields - fixed bugs in pt-server (lockup, unknown species just after building pt-server) - improved performance during pt-server-build - several programs coming along with ARB where updated (PHYLIP,...) - reads EMBL genom files - support for experiments (genom databases only) Major changes in Beta 2001_07_24 (24 Jul 2001): - basic support for genoms (Gene Map, reads Genebank files) - ported to libc6