This are quick notes for myself. Just an experimenting posting my notes publicly.


data science

Installation of Octave on OS X Catalina 10.15.2

26 January 2020

Prerequisites

I prefer Homebrew as package manager Installed as per brew.sh Homebrew requires xcode and for xcode license to be accepted.

Installation

This will take the latest version available as a package

# list available
brew search octave
# isntall latest (5.1.0_8 at time of writing)
brew install octave

Installs many dependencies but most are keg-only dependencies (i.e. not symlinked to /usr/local so visible only to the other homebrew apps) with many caveats (read the caveats, nothing bad initially). lso installs python3 if not present.

% which python3
/usr/local/bin/python3
# confirmed it's using homebrew's install

Quick installation tests

$> octave

octave:1> x=linspace(0,1)
octave:2> y=2*x
octave:3> plot(y)

Verify that plot’s window opens okay.

IntelliJ React Native - Unhandled JS Exception - Maximum call stack size exceeded

26 November 2016

I wasted several hours tying to isolate this problem so I’m posting quickly for others.

Unhandled JS Exception: Maximum call stack size exceeded
...
in index.ios.bundle during use of `regenerator-runtime/runtime` with a getValue() method

This error is caused by infinite recursion during initialization. It occurs when using IntelliJ IDEA Ul 2016.3 and react-native 0.37 or 0.38.

The cause is tracked by IntelliJ WEB-24149, discussed (and dismissed) as a node bug.

At the time of writing the only work-around was to downgrade to react-native 0.36.1 instead of 0.37/0.38.

To downgrade:

  1. Verify that package.json doesn’t enforce any newer versions. I had to downgrade react-native, react and react-test-renderer:
    "dependencies": {
     "react": "15.3.1",  
     "react-native": "0.36.1"  
      },
      "devDependencies": {    
     "react-test-renderer": "15.3.1" 
      }
    
  2. Clean up the node modules (shouldn’t be necessary but was)

    rm -rf node_modules/ && npm install

  3. Re-initialize the project using react-native at the downgraded version

    react-native init <projectname> --version react-native@0.36.1

  4. Clean up other caches when starting the packager:

    watchman watch-del-all && npm start -- --reset-cache

Bootstrapping a new hapi.js project

9 June 2015

Whenever I start a new node.js project there’s a bit of mucking around to get the all the boilerplate just right.

mkdir newproject
git init

I’m using bitbucket for private repositories. Create the repo there first.

git remote add origin git@bitbucket.org:jeromyevans/newproject.git

vi README.md
git add README.md

Setup package.json and make it private

npm init  #setup package.json
vi package.json

Add the key:
“private”: true

git add package.json

Setup hapi and grint

npm install hapi --save
#npm install -g grunt-cli      (usually already available)
npm install grunt --save-dev

I don’t have a good way of seeding a Gruntfile yet, so I copy it from another source I always use a jshint task in grunt, so include that dev dependency

npm install grunt-contrib-jshint --save-dev

Create a .jshintrc from a good source (jshint options) vi .jshintrc git add .jshintrc

Create a .gitignore from a good source: node.js: https://github.com/github/gitignore/blob/master/Node.gitignore intellij: https://github.com/github/gitignore/blob/master/Global/JetBrains.gitignore

vi .gitignore
git add .gitignore

Depending on the app, I’ll use grunt-nodemon to watch node src files npm install grunt-nodemon –save-dev

AusKey Read-only files cannot be used as keystores in OS X

7 May 2015

I seem to get this error every time I try to use the ATO website from OS X. It states that Read-only files cannot be used as keystores with a reference to a keystore.xml file in the user’s local application support directory.

TL;DR;

Within Safari, go to Preferences->Security->Website Settings->Java Plugin. Then give the authentication.business.gov.au website to run Java in “Unsafe Mode”. The site will be listed because you’ve already authorised the plugin to run for the site.

Error Details

This error isn’t a filesystem permission error. It’s a Java plugin privilege error, specifically that the AusKey Java Applet needs explicit permission to read/write a file from your local filesystem. This is disabled by default for all Java applets.

There’s basically three levels of permission:

  1. The Java runtime is installed and the Java plugin enabled in Safari
  2. That you’ve been to the website and authorised the Java applet to run (which starts but fails)
  3. The ATO Business Portal website is authorised to run the plugin in Unsafe mode (given access to the filesystem).

How this isn’t listed in the AusKey Troubleshooting FAQ, I don’t know.

Step 1 is resolved after installing Java by going to the System Preferences -> Java Control Panel -> Security and enabling Java in the Web Browser (checkbox)

Step 2 is resolved by attempting to login to an AusKey website and authorizing the Java plugin to run

Step 3 is resolved in Safari Preferences->Security->Website Settings->Java Plugin. The authentication.business.gov.au website will be listed and needs to be changed from “Allow” to “Run in Safe Mode” (for this site only).

This gist is a step-by-step sequence to fix it.

Scaffolding for a javascript project

16 April 2015

create new for the project directory

$ mkdir project $ cd $! $ git init

Use npm init to create a new package.json file

$ npm init

Input grunt and add as a development dependency in package.json

$ npm install grunt –save-dev

get a decent .gitignore file

$ wget https://raw.githubusercontent.com/github/gitignore/master/Node.gitignore -O .gitignore

Create the project on github $ hub create

Using bower? Using jasmine for testing? bower install jasmine

get a starting gruntfile

https://github.com/gruntjs/grunt-contrib-jasmine

Install phantomJS for headless testing

dns aws

Route 53

11 January 2015

I switched the DNS hosting for this site over to AWS’s Route 53 today. It couldn’t be simpler, with a control panel with more options/controls han the previous host.

Instructions to set it up: AWS Static Hosting Walkthrough including Route 53

osx development

Font Awesome

25 November 2014

Source of icon fonts used by this site: Font-Awesome

osx development

Jekyll installation

10 November 2014

This website used to be hosted on an EC3 node using wordpress. I didn’t really maintain it and was getting tired of it going down because Wordpress and PHP are so vulnerable to hacks if not maintained regularly. Instead of moving to a dedicated wordpress host I thought I’d experiment with static site hosted on S3.

Motivation

  • S3 is cheap. A minimal EC2 node would cost $15 per month. The minimal cost to host static data on S3 with CloudFront is lower.
  • There’s good tools for generating static sites and posting to S3. I decided on Jekyll because I’d seen the Jykll + S3 + CloudFront combination mentioned before:

This website is built using Jekyll and deployed straight to S3 using s3_website and cloudfront.

Prerequisites

    gem install jekyll

Then create the new project

    # pooloferrors is the name of the site. It'll create a subdirectory with default layout
    jekyll new pooloferrors

The default directory layout and configuation of jekyll is straight-forward.

Publishing to S3

The s3_website is used the static site to S3. The script configured the S3 bucket so all files are publicly readable.

   
gem install s3_website
s3_website cf create

Set up the minimal config to include the AWS credentials with a role that can write to S3.

Publish the site with push:

s3_website push

Notes

  • Markdown is great, but also very restrictive if you want to get creative with the CSS. I haven’t used Textile but it seems to provide more layout options. I’m trying to focus on the content so I’ve stuck with markdown. Converting old wordpress blog posts to markdown was simple (but manual)
  • The datestamp within the front matter of posts is more important than the datestamp in the filename. They need to be accurate to the date, and these are the values referenced by post_ref.
  • The index.html and other files can be customised to support multi-site using the [Liquid Template[(http://www.rubydoc.info/gems/liquid) language. A simple of multisite example is Create a Multi Blog Site with Jekyll
  • The engine supports pagination, but the built-in paginator doesn’t understand filtering in the index files.

AWS Walkthrough

javascript node.js

restify

10 October 2014

restify looks like a good node framework for rest APIs http://mcavage.me/node-restify/ but there’s a lot of comments that it’s slow (least of my problems)

npm install restify —savedev

client side sounds like ember is best today

It’s good, but it’s dead/incomplete compared to express.js

javascript node.js

async

10 October 2014

POSTing using curl

curl -X POST "http://localhost:8080/projects”

curl -X POST "http://localhost:8080/accounts -H 'connection: close'

(by default, curl uses connection: keep-alive)
javascript node.js

node validators

10 October 2014

Lacking a decent parameter validator npm install validator –save-dev

https://github.com/chriso/validator.js

express-validator did not seem to work with testify (tried it)

restify-validator does not seem to be well maintained, but at last its using node-validator

javascript node.js

uuid

10 October 2014

generating a uuid

https://www.npmjs.org/package/node-uuid

npm install node-uuid

javascript node.js

passport

10 October 2014

=== the Passport middleware for express looks like the best authentication approach npm install passport —save-dev

http://passportjs.org/guide/configure/

for http basic npm install passport-http –save-dev

javascript node.js

oauth2

10 October 2014

oAuth2 http://scottksmith.com/blog/2014/07/02/beer-locker-building-a-restful-api-with-node-oauth2-server/ using http bearer tokens for authorisation Authorization: Bearer npm install passport-http-bearer --save-dev

npm install oauth2orize —-save-devv

oauth2orize requires a Session Hopefully express-session works.

javascript node.js

bycrypt

10 October 2014

bcrypt for hashing https://www.npmjs.org/package/bcrypt-nodejs

javascript node.js

async

10 October 2014

The aysnc library helps prevent node nested callback anti patterns (remember I/O is async) npm install async –save-dev https://github.com/caolan/async

async.series([functions…],finalCallback)

javascript node.js

MogoDB

10 October 2014

mongodb is great. It’s refreshing. mongolab looks sufficient for prod (if delay okay) heroku for node? static site on s3

== mongodb

brew update brew install mongodb (need rwx permission on /usr/local/var)

create the data directory. default is /data/db. Is isn’t good for development mkdir -p ./data/db

start mongo mongod -dbpath ./data/db

WARNING: soft rlimits too low. Number of files is 256, should be at least 1000

Run a mongo client: mongo use mm-dev List all projects: db.projects.find()

mongoose for models

javascript

Pubsub

10 October 2014

needed a pub/sub on the client side. This seems to do the trick. https://github.com/mroderick/PubSubJS bower install pubsub-js

<== this was no good (that’s the version suitable for node). I had to create the jquery build

http://roderick.dk/resources/using-pubsubjs-with-jquery/

Although the jquery build offers nothing special? Not worth creating a custom builder.

javascript

Highcarts

10 October 2014

There’s an official shim on github https://github.com/highslide-software/highcharts-release

bower install highcharts-release

osx development

Assemble notes

10 September 2014

Assemble is a static site generator that works well with [grunt]. It’ll parse .md or other templates as a grunt task and output generated site files. It’s like [jekyll] except generic. Assemble is suited to generic arbitrary static pages for a webapp, while jekyll is great for post-oriented sites (blogs).

Each page is written in hbs (handlebars). The YAML at the top of each page can specific data for the template, such as which layout to use. It can use includes (other hbs).

http://www.objectpartners.com/2014/02/26/running-a-lightweight-static-server-with-grunt/

Use grunt-contrib-connect for a simple local static web server

npm install grunt-contrib-connect –save-dev

add the task to Gruntfile.js register the connect task

Connect live-reload can be used ??

I don’t like the way assemble customised bootstrap. I’m going to start with a client installation

Just realised assemble uses assemble-less instead of grunt-contrib-less. Best to stick with the original.

Switch to the standard npm install grunt-contrib-less —save-dev

osx development

Yeoman Notes

9 September 2014

Scaffolding

Yeoman defines scaffolding for modern web applications.
Generators are used to start projects based on specific archetypes (used with grunt and bower). It’s much like the maven archetype plugin for Java projects.

npm install -g yo

I don’t like it. The templates I reviewed assume or prescribe too much. I prefer to start with minimal scaffolding and build it up. Stick with grunt-init

osx development

Grunt installation on OS X

2 September 2014

Grunt is a node.js build to. It’s generally useful for build automation in any javascript project.

Prerequisites

  1. Install homebrew first
  2. install node.js first

Installation

npm install -g grunt
npm install -g grunt-cli

Install the official grunt-init-node template to create node modules

npm install -g grunt-init

git clone https://github.com/gruntjs/grunt-init-node.git ~/.grunt-init/node

grunt-init node
osx development

Bower installation on OS X

2 September 2014

Bower provides package management and distribution for common front-end packages.
The distributions are not as consistent as other packages as you still have to manually choose which sub-dirs or files are copied into your deployment, but for version and dependency management it’s great.

Prerequisites

There isn’t much to it. Just install bower globally with npm.

npm install -g bower

Then use bower to install packages as required.

osx development

Node.js installation on OS X

2 September 2014

Prerequisites

  1. Install homebrew first

Installation

    brew install node

    # create a NODE_PATH variable in the env to provide an additional location to search for modules
    echo "export NODE_PATH=$NODE_PATH:/usr/local/lib/node_modules" >> ~/.bashrc && source ~/.bashrc

npm install will install the project dependencies specified in package.json. ie. if you clone a node project, the first thing to do is npm install

    npm install
osx development

Ruby installation on OS X

2 June 2014

Prerequisites

  1. Install homebrew first
brew install ruby
🍺  /usr/local/Cellar/ruby/2.1.2_2: 942 files, 20
   
ruby -v
ruby 2.1.2p95 (2014-05-08 revision 45877) [x86_64-darwin13.0]
osx development

Homebrew installation

1 June 2014

Install Homebrew

Homebrew provides distribution and package management for common developer packages. It avoids using ad-hoc, developer-specific set ups and version conflicts. Almost everything you need to support java development, ruby and node.js is available.

    ruby -e "$(curl -fsSL https://raw.github.com/Homebrew/homebrew/go/install)"

    # Add the Homebrews binary path to the front of the $PATH
    echo 'export PATH=/usr/local/bin:$PATH' >> ~/.bash_profile
    source ~/.bash_profile

Check the installation with brew doctor.

    brew doctor

When I originally installed homebrew I had errors because I use the mac as multiple users (one for work work, one for other work).

    ! brew install problem
    ! Configure brew to allow other users to write to Cellar      
    
    Larson-2:~ larson$ brew install postgresql
    Error: Cannot write to /usr/local/Cellar
    
    Larson-2:~ larson$ sudo brew install postgresql
    Cowardly refusing to `sudo brew install'

    Jeromys-MacBook-Pro:~ jeromyevans$ ls -ld /usr/local/Cellar
    drwxr-xr-x  8 jeromy  admin  272 22 Jun 21:13 /usr/local/Cellar
    sudo chmod -R g+w /usr/local/Cellar
    Jeromys-MacBook-Pro:~ jeromyevans$ brew install node
    Error: Permission denied - /Library/Caches/Homebrew/Formula/node.brewing
    sudo chmod -R g+w /Library/Caches/Homebrew
    sudo chmod -R g+w /usr/local/

Bring brew up to date. This may be need to be repeated from time to time.

    brew update

And try it out by installing node.js

    > brew install node   
    🍺  /usr/local/Cellar/node/0.10.29: 1547 files, 18