UFO Core 0.10 Released

Our work continues with UFO Core 0.10 which is to bring major improvements to the code as listed below.

IMPORTANT: eHRC – Hard Fork – Block 1,182,000

NSGminer 0.9.3

Please note that UFO 0.10 removes the getwork RPC call from the client. getwork allowed external miners to find a suitable hash but left block creation up to the client. 0.10 uses getblocktemplate which lets the miner create the block. The NSGminer has been updated to work with NeoScrypt coins using getblocktemplate, to solo mine on the 0.10 release you will need to upgrade to NSGminer 0.9.3 and make sure that you provide an address to mint coin to using the --coinbase-addr option.

Faster synchronization

UFO Core now uses ‘headers-first synchronization’. This means that peers are asked for block headers first and validate those. In a second stage, when the headers have been discovered, the blocks are downloaded. However, they can be downloaded in parallel from all available peers.

In practice, this means a much faster and more robust synchronization. You may notice a slower progress in the very first few minutes, when headers are still being fetched and verified, but it should gain speed afterwards.

A few RPCs were added/updated as a result of this:

getblockchaininfo now returns the number of validated headers in addition to the number of validated blocks.
getpeerinfo lists both the number of blocks and headers we know we have in common with each peer. While synchronizing, the heights of the blocks that we have requested from peers (but haven’t received yet) are also listed as ‘inflight’.
A new RPC getchaintips lists all known branches of the block chain, including those we only have headers for.

Dynamic transaction fees

This release automatically estimates how high a transaction fee (or how high a priority) transactions require to be confirmed quickly. The default settings will create transactions that confirm quickly; see the new ‘txconfirmtarget’ setting to control the tradeoff between fees and confirmation times. Fees are added by default unless the ‘sendfreetransactions’ setting is enabled.

Prior releases used hard-coded fees (and priorities), and would sometimes create transactions that took a very long time to confirm.

Statistics used to estimate fees and priorities are saved in the data directory in the fee_estimates.dat file just before program shutdown, and are read in at startup.

New command line options for transaction fee changes:

-txconfirmtarget=n : create transactions that have enough fees (or priority) so they are likely to begin confirmation within n blocks (default: 1). This setting is over-ridden by the -paytxfee option.
-sendfreetransactions : Send transactions as zero-fee transactions if possible (default: 0)

New RPC commands for fee estimation:

estimatefee nblocks : Returns approximate fee-per-1,000-bytes needed for a transaction to begin confirmation within nblocks. Returns -1 if not enough transactions have been observed to compute a good estimate.
estimatepriority nblocks : Returns approximate priority needed for a zero-fee transaction to begin confirmation within nblocks. Returns -1 if not enough free transactions have been observed to compute a good estimate.

REST interface

A new HTTP API is exposed when running with the -rest flag, which allows unauthenticated access to public node data.

It is served on the same port as RPC, but does not need a password, and uses plain HTTP instead of JSON-RPC.

Assuming a local RPC server running on port 8332, it is possible to request:

Blocks: http://localhost:8332/rest/block/HASH.EXT
Blocks without transactions: http://localhost:8332/rest/block/notxdetails/HASH.EXT
Transactions (requires -txindex): http://localhost:8332/rest/tx/HASH.EXT
In every case, EXT can be bin (for raw binary data), hex (for hex-encoded binary) or json.

For more details, see the doc/REST-interface.md document in the repository.

Improved signing security

For 0.10 the security of signing against unusual attacks has been improved by making the signatures constant time and deterministic.

This change is a result of switching signing to use libsecp256k1 instead of OpenSSL. Libsecp256k1 is a cryptographic library optimized for the curve Bitcoin uses which was created by Bitcoin Core developer Pieter Wuille.

There exist attacks[1] against most ECC implementations where an attacker on shared virtual machine hardware could extract a private key if they could cause a target to sign using the same key hundreds of times. While using shared hosts and reusing keys are inadvisable for other reasons, it’s a better practice to avoid the exposure.

OpenSSL has code in their source repository for derandomization and reduction in timing leaks that we’ve eagerly wanted to use for a long time, but this functionality has still not made its way into a released version of OpenSSL. Libsecp256k1 achieves significantly stronger protection: As far as we’re aware this is the only deployed implementation of constant time signing for the curve Bitcoin uses and we have reason to believe that libsecp256k1 is better tested and more thoroughly reviewed than the implementation in OpenSSL.

[1] https://eprint.iacr.org/2014/161.pdf

Watch-only support

The wallet can now track transactions to and from wallets for which you know all addresses (or scripts), even without the private keys.

This can be used to track payments without needing the private keys online on a possibly vulnerable system. In addition, it can help for (manual) construction of multisig transactions where you are only one of the signers.

One new RPC, importaddress, is added which functions similarly to importprivkey, but instead takes an address or script (in hexadecimal) as argument. After using it, outputs credited to this address or script are considered to be received, and transactions consuming these outputs will be considered to be sent.

The following RPCs have optional support for watch-only: getbalance, listreceivedbyaddress, listreceivedbyaccount, listtransactions, listaccounts, listsinceblock, gettransaction. See the RPC documentation for those methods for more information.

Compared to using getrawtransaction, this mechanism does not require -txindex, scales better, integrates better with the wallet, and is compatible with future block chain pruning functionality. It does mean that all relevant addresses need to added to the wallet before the payment, though.

Consensus library

The purpose of this library is to make the verification functionality available to other applications, e.g. to language bindings such as python-bitcoinlib or alternative node implementations.

This library is called libbitcoinconsensus.so (or, .dll for Windows). Its interface is defined in the C header bitcoinconsensus.h.

In its initial version the API includes two functions:

bitcoinconsensus_verify_script verifies a script. It returns whether the indicated input of the provided serialized transaction correctly spends the passed scriptPubKey under additional constraints indicated by flags
bitcoinconsensus_version returns the API version, currently at an experimental 0
The functionality is planned to be extended to e.g. UTXO management in upcoming releases, but the interface for existing methods should remain stable.

Relaxed P2SH address rules

The IsStandard() rules have been almost completely removed for P2SH redemption scripts, allowing applications to make use of any valid script type, such as “n-of-m OR y”, hash-locked oracle addresses, etc. While the Bitcoin protocol has always supported these types of script, actually using them on mainnet has been previously inconvenient as standard Bitcoin Core nodes wouldn’t relay them to miners, nor would most miners include them in blocks they mined.

UFO-tx utility

It has been observed that many of the RPC functions offered by bitcoind are “pure functions”, and operate independently of the bitcoind wallet. This included many of the RPC “raw transaction” API functions, such as createrawtransaction.

bitcoin-tx is a newly introduced command line utility designed to enable easy manipulation of bitcoin transactions. A summary of its operation may be obtained via “bitcoin-tx –help” Transactions may be created or signed in a manner similar to the RPC raw tx API. Transactions may be updated, deleting inputs or outputs, or appending new inputs and outputs. Custom scripts may be easily composed using a simple text notation, borrowed from the bitcoin test suite.

This tool may be used for experimenting with new transaction types, signing multi-party transactions, and many other uses. Long term, the goal is to deprecate and remove “pure function” RPC API calls, as those do not require a server round-trip to execute.

Other utilities “bitcoin-key” and “bitcoin-script” have been proposed, making key and script operations easily accessible via command line.